For those of you who aren’t aware, most UK-based insurance firms (especially those applying to use their own internal models) will currently be concentrating on the FSA Solvency 2 ‘Data Audit’ and whilst they will all be at various stages in the process, they will be focusing on similar pieces of work and experiencing similar challenges. The ‘Data Audit’ is effectively a review requested by the FSA in which each firm is expected to undertake an independent audit (internal, external or a mixture of both) of their data management practices. The findings of this audit will subsequently form part of the FSA’s Internal Model Approval Process (IMAP) and help the FSA in its assessment of whether a firm is compliant with the standards for data as set out in the Solvency 2 directive. The scope of the review has been defined as all data (both internal and external) that could materially impact the Internal Model.

As part of their supportive material the FSA developed a review ‘tool’ to be used as guidance in undertaking the review. It is basically a short document that informs firms on what they could/should be doing in order to satisfy the standards set out in the Solvency 2 directive and also the expectations as to what could be required as supportive evidence to a firm’s application.

After the review is completed the FSA expect the organisation to compile and submit a summary report of findings and be in a position to make available any supportive evidence.

The review schedule has five sections as follows:

1. The approach to managing data used in the internal model (i.e. a data policy)
2. The level of oversight around the development and implementation of the data policy
3. The level of understanding of data used in the internal model
4. The impact of data issues on the integrity of the internal model and management decisions
5. General IT issues that could compromise the quality of data of the internal model

These sections correspond to the risks that the FSA have called out as being key considerations in ensuring that data used in the internal model meets the data quality requirements of the Solvency 2 directive. Whilst the risks are relatively high level, for each one the FSA have also detailed their expected controls. It is this information along with the suggested assessment approach which, I feel, is some of the most prescriptive and useful guidance we’ve had as to what should be in place for Solvency 2. These risks and expected controls can be summarised as follows:

Risk 1: The approach to managing data for use in the internal model does not ensure consistency in quality and application of the internal model.

Expected controls: An established data policy with relevant procedures and standards. The data policy should as a minimum contain: defined data sets; a definition of materiality; ownership, roles and responsibilities; definition of data quality assessment; process for the use of assumptions; process for data updates to the internal model; and a process for undertaking risk and impact assessments.

Risk 2: Inadequate oversight of the development and implementation of the data policy increases the risk of poorly informed decision-making and non-compliance with the required quality and standards.

Expected controls: Defined and operational Data Governance structures and processes. A system for reporting data quality metrics and a process for the management of data deficiencies.

Risk 3: Lack of a clear understanding of the data used in the internal model, and of its impact and vulnerabilities, can create gaps in ownership and control.

Expected controls: A directory of all data (or a ‘data directory’ as many firms are referring to it as) used in the internal model specifying source, usage and characteristics. Also a completed risk and impact assessment calling out the impact of poor quality data, where failures are more likely to occur and tolerances for when data related issues become material.

Risk 4: Errors, omissions and inaccuracies in the data can undermine the integrity of the internal model and management decision making.

Expected controls: Implementation of data quality controls that include checks for the Solvency 2 definition of ‘completeness’, ‘accuracy’ and ‘appropriateness.

Risk 5: Unreliable IT environment, technology or tools can compromise the quality and integrity of the data and its processing within the internal model.

Expected controls: IT general computer (ITGC) controls such as: access management; change management; IT security; business continuity; and incident management.

For most firms the list of expected controls will pose a fair challenge and, depending on how seriously they have taken data management in the past, there will be any number of gaps to address.

As I stated, I believe this guidance is the some of clearest and most prescriptive to come out from the FSA with regards to the data requirements of Solvency 2; however it is still not entirely free from subjective interpretation. Many of the challenges I have experienced are initially caused by the differing interpretations of stakeholders as to what is actually required. It’s a case of striking the correct balance between the expectations of the FSA, the auditors and the various internal stakeholders whilst also trying to deliver business benefit (or at least trying to minimise operational impact). There’s also the (always fun) challenge of meeting tight timescales, resource constraints and ever-changing requirements; which means establishing the sweet spot of delivering flexible solutions that meet the requirements of Solvency 2 without ending up being convoluted, unachievable or of inhibiting expense. In summary, there is a lot of work to do!

Hopefully a lot of the UK banking firms are taking serious note and following the developments of the insurance firms participating in Solvency 2. The reason I say this is that Basel 3 (Basel III) is coming round the corner and I would imagine that there will be very similar data-related directives coming with it. Of course the smart firms will already have kicked things off; won’t they?

Unfortunately the answer is yes. The bigger problem though is that it’s not just CIOs who think like this. Although they may hold the purse strings when it comes to application investment, they can be swayed as they look to their trusted advisors for guidance. But if the general opinion that Excel or some other EUC solution is ‘probably good enough’ then someone trying to push through a more strategic and robust solution could be facing an impossible task.

People are becoming more technologically savvy and as a result a new wide-spread confidence in data related tasks can be seen. In some ways this is a good thing as a better general awareness of data related issues can aide in the implementation of governance and quality initiatives. However, it can also mean that business areas are much more likely to  bypass the traditional IT development processes in order to implement their own EUC solution and especially so if they have been previously burned by IT. A new breed of DIY data solutions rise up in silos across an organisation thanks in part to the ease at which tools like Excel make basic data management accessible to the masses.

What this all means is that a strong and well implemented Data Governance programme is needed more than ever to ensure that appropriate controls, processes, people and technology are in place. EUC solutions most definitely have their place, but they must be used and controlled appropriately and be covered by the same governance as any other technology within the Enterprise or data management issues can proliferate and escalate rapidly.

]]> http://www.garryure.co.uk/2012/04/06/diy-data-management/feed/ 1 http://www.garryure.co.uk/2012/02/23/destination-unknown/ http://www.garryure.co.uk/2012/02/23/destination-unknown/#comments Thu, 23 Feb 2012 13:07:22 +0000 Garry http://www.garryure.co.uk/?p=304 Continue reading →]]>

There was an excellent discussion over at Henrik Liliendahl Sørensen’s ( @hlsdk ) blog recently where a debate opened up over whether the concept of data quality should be best likened to a journey or a destination. The more commonly recognised metaphor is that data quality is indeed a journey, however John Owens ( @JohnIMM ) argued that, in fact, data quality is actually a destination. His point being that in considering it an endless journey would in fact be akin to giving “DQ practitioners, and their grandchildren, a job for life”. He stressed that “Quality data should be created as an integral part of doing business day-to-day”. Now I admitted that I could logically agree with both sides of the argument; historically the quest for data quality was likened to a journey to convey the concept that you need to continue to work in order to maintain quality. However I can also see the danger in creating a data quality ‘cottage industry’ which can actually hamper the evolution of a quality improvement culture. Why would the business need to concern themselves as much with quality if they knew there was a dedicated function to sort it out for them? In a similar way though I feel that by considering the ‘journey’ over, even though you have successfully ingrained quality practices into day-to-day operations, sends out the wrong message. I would class John’s notion of DQ as part of integral business processes as just one destination of many on a long and somewhat recursive journey. In fact there are numerous journeys each with their own destinations (or perhaps stations is a better term to use, maintaining the concept of a journey). I think the point should be that there is no “final destination”, instead the journeys become smoother, quicker and more pleasant for those travelling.

Contrary to John’s argument regarding “a job for life” I believe maintaining quality data is a job for life. However I completely agree that it is vital that activities transition into integral BAU processes. This though, does not mean that focus on quality improvement can wane. It should be the case that DQ practitioners are able to disembark leaving BAU staff to continue the journey.

I like to compare the inception of a regulatory project with the fuelling of an aeroplane. You know where your destination is and you can work out how much fuel you need to get there, but would you ever think of fuelling the plane with the minimum amount required to get you to your destination? Imagine if you flew into difficulties, had a stronger head wind than predicted or couldn’t land at the planned airport. You would always want to have enough fuel to cover all eventualities, as the last thing you’d want to happen is to fall short (literally). However, as with aspirational projects, you don’t want to carry far too much fuel as this would cause it’s own problems. So you need to find the right balance of risk and reward.

The same goes for projects; if you decide that your destination is the closest possible place that would make you compliant and fall short then you are potentially in a heap of trouble. Likewise, should the regulator change the goalposts and increase the requirements for compliance, you could find yourself having a major re-planning exercise mid-project. So how do you find the ‘happy place’ between aspiration and falling short?

I think focusing only on the minimum is a very pessimistic, short-sighted and unambitious approach for an organisation to take. It’s fine to consider it a stop-over on your journey but please don’t make it your final destination and be sure to have enough fuel (be that budget, resource or strategic vision) to get you where you want to be. Nicola Askham recently wrote about this situation in her paper for IRM. In it she recommends taking a phased approach where the first phase should deal with achieving the foundation of compliance and subsequent phases used to build on this foundation. So again to reinforce the point, the minimum may well be part of your journey where you can stop and refuel, but it should not be your final destination. Breaking the journey into smaller steps allows you approach it in a much more manageable way and gives you natural points to reflect on where you have come from and where you are heading next. That way if you do find yourself having to replan your route, it is much more easy to do so.

From my point of view, 2011 was a pretty good year all things considered. There were plenty of ups and downs, both from the personal and professional aspects of my life but I think, in retrospect, the downs were outnumbered by the ups.

I spent the entirety of 2011 working with a single organisation on their Solvency II programme and although challenging, this gave the year a degree of stability and consistency. In the latter part of the year I began to focus on improving my professional network and as a result I have met (albeit in most cases, virtually) some fantastic people and found the DQ community to be extremely talented, friendly and welcoming. I published my blog and stepped up my use of Twitter and I now feel that I am starting to make some progress with regards to marketing my professional services.

So what do I want to focus on in 2012?

• Learning - I already use every opportunity to expand my knowledge of all things DQ as well as improve my knowledge of related business and technical subjects, so 2012 will be no different in this respect. However this year, I may also be tempted to consider a formal accreditation to help improve my credibility to clients and help bolster my own self-belief. The wealth of information and knowledge sharing by the online communities these days is truly priceless and it means you always have access to high quality  learning materials.
• Experiencing - I stated in my DataQualityPro interview last year that I was keen to increase my experience in other industry sectors and this is something I’d certainly consider this year. Although I have been able to move from banking in my previous role to insurance with my current role, I’d be interested to experience life outside of the financial sector.
• Networking - I plan to put effort into expanding my network this year and will try to participate and contribute to the online community more. The rise in social media has been a massive advantage for me and I think anyone not involved in building an online presence (or at least benefiting from the vast amount of online resource) is really missing out.
• Building - I want to spend some time working to improve my own ‘toolkit’. By ‘toolkit’ I mean my own collection of processes and checklists for things like client engagement, data profiling, stakeholder management etc. Also, all the good reference materials like generic DQ frameworks, Governance models and various templates for standard documents that I can dip into and will allow me to hit the ground running with a new client.
• Living - Very important to me is to ensure that I am always in a position to maintain a healthy work/life balance.

Here’s to a great quality 2012!

It’s not that it’s a difficult question to answer (although attributing costs and impacts can be challenging) it’s just that I can’t help but find this question at times a lazy and ignorant response to the raising of a data related issue. It often belittles a problem and inadvertently endorses a culture that is largely indifferent to data.

Maybe it again comes down to the fact that data is not tangible and that senior management, as a general collective, are not culturally or practically used to thinking about data related matters. (Perhaps this will change in time, as there is a gradual generational shift)

What reaction would follow news related to a tangible (and more easily valued) asset? Say there was a fire in a property; I doubt that the immediate response would be “so what?” It would more likely be a machine-gun set of questions: Was anyone hurt? How bad was it? Is it still operational? What’s the impact? What’s the cost to repair? Does our insurance cover it? How did it start? How do we stop it happening again? Have we checked all our other properties?

All valid questions and all equally valid in relation to a data related issue, so why the difference in reaction?

It could of course be down to the way the issue is presented. The challenge is in the common default fascination for managers to have endless metrics and evidence without the understanding of what it is telling them. As I stated in my previous post, there is a natural tendency to take data for granted and by focusing on the minutiae of metrics people can get lost and confused and consequently miss the point; cue the “so what?”

You need to force people to step back for a minute and think about things more conceptually; step back to a level where you are sure they understand the impact and then you can bring it slowly forward again in order to properly quantify and analyse the issue. As with the fire there will be warning signs: the smell of burning; or the sight of smoke long before you feel the heat. Of course it would be easier if management could also understand these signs, but they often don’t and therefore the controls and alarms you put in place need to clearly convey the danger in a way that is understandable to all.

We need to be able to turn the “so what?” into the “tell me more”.

Any mature organisation that uses data will know that, data can simultaneously cost you money and save (or make) you money through-out it’s lifecycle. The enduring challenge is ensuring that you can overcome the associated costs and tip the balance in favour of the saving/making money side. Danette McGilvary states, in her book, that all stages of the data lifecycle have associated costs and by continuing the illustrative analogy of balancing a set of scales we can begin to show what factors in this lifecycle can affect the equilibrium.

On one side of the scales you have the costs associated with things like data storage, software/hardware costs, maintenance overheads and the cost of resources required for data capture. In order to counter-balance these costs we need to ensure that the other side of the scales hold enough weight and we can establish ways to actually reduce weight on the costs side.

As part of your balancing act you need to ensure that your data assets are appropriately managed throughout the data lifecycle. This will involve finding ways to reduce storage (think MDM and SCV), reducing the number of occasions that the obtaining of data is duplicated across the organisation and most importantly use and reuse the data more. A key tactic will be to unearth new stakeholders and new uses for data assets.

Conceptually it is easy to see how you can balance cost and savings but actually working out what the values on both sides are can be incredibly difficult. Tangible costs and savings are easy to quantify however it’s the unseen costs and savings that can often tip the scales one way or the other. For example, until now I haven’t even mentioned the quality of your data; we’ve assumed that the data is of sufficient quality to help tip the scale but what if you have quality problems?

As you’d expect poor data will not help you save money; it will in fact tip the scales the other way. In the same way that a defective component will impact the quality of the end product in a manufacturing process, poor data will impact outputs and cost you additional funds to correct. The more you reuse poor quality data, the more it costs you in the longer term. All those ‘shadow’ processes that have secretly crept into business areas in order to fix or compensate for bad data cost you and ultimately impact the actual value of your data assets. Your data has a potential value that can only be realised when the organisation is using and managing it in an optimal way.

This post obviously tries to very briefly cover a very small part of a very large subject that would easily fill a book (and indeed has filled many), but I thought it would be useful to start the topic and maybe revisit it with later posts. I’ll leave you with some basic concepts to think about:

• Good quality data saves or makes you money every time it is used (assuming appropriate use).
• Bad quality data costs you money every time it is used.
• Each stage in the data lifecycle costs money. Try to optimise and reduce duplication in each stage.
• Promote and increase the use and reuse of the same data across the organisation.

As always, thoughts and comments encouraged.

I’m a big fan of ‘not reinventing the wheel’, when there is no real need to, and always look for methods and tools that have been successfully utilised by others in the past. Having previously read Arkady Maydanchik‘s book Data Quality Assessment, I was keen to use the same rule categories that he proposed. For those that haven’t read Arkady’s book I’ll note the categories and sub-categories that he suggests but for the more detailed explanations I’d recommend buying Arkady’s book (I’m not on commission, I promise) or checking out his articles at DataQualityPro.com and IAIDQ.

Arkady suggests that DQ rules should fall into one of the five broad categories outlined below:

1. Attribute Domain Constraints
2. Relational Integrity Rules
3. Rules for Historical Data
4. Rules for State-Dependent Objects
5. General Attribute Dependency Rules

Within each of these categories are more specific sub-categories that allow more precise categorisation of rules (there are also sub-sub-categories, but I’ll leave them out here):

1. Attribute Domain Constraints
• Optionality Constraints
• Format Constraints
• Valid Value Constraints
• Precision Constraints
2. Relational Integrity Rules
• Identity Rules
• Reference Rules
• Cardinal Rules
• Inheritance Rules
3. Rules for Historical Data
• Timeline Constraints
• Timeline Patterns
• Value Patterns
• Rules for Event Histories
4. Rules for State-Dependent Objects
• Domain Constraints
• Transition Constraints
• Timeline Constraints
• Advanced State Dependant Rules
5. General Attribute Dependency Rules
• Attribute Redundancy
• Attribute Dependency
• Partial Dependency
• Attribute Correlation

What you should find is that any DQ rule you can come up with, fits nicely into one of the above categories. This therefore allows you to group rules together in a more meaningful way and, post data assessment, helps you formulate the grouping of erroneous records for reporting and remediation.

Cataloguing the rules in this way also ensures that you (and your colleagues) are working within a structured and consistent framework that you can use as a reference when defining DQ rules in the first place. It is important to use it as a kind of aide-mémoire to ensure that you are considering all categories for each data element.

Lastly, I would also recommend having your own specific examples for each category of rule. I have found that this becomes an invaluable training/education/communication resource to have at your disposal.

Happy cataloguing!

For those of you who haven’t seen an episode of ‘House’ I’ll provide a generic plot breakdown:

• Someone is brought to hospital with an unknown or rare ailment.
• Dr Gregory House is brought in because no one knows what to do; his skilled (and long-suffering) diagnostics team detail the symptoms to House and offer a diagnosis (usually lupus).
• House is rude to everyone and makes a flippant diagnosis of his own.
• Everyone disagrees with House but they start treatment anyway.
• The treatment doesn’t work and the patient gets sicker. House gets angry and his diagnostics team suggest a number of other diagnoses.
• House is rude to more people and ignores all other diagnoses.
• In the middle of insulting someone House has a moment of clarity/inspiration and comes up with the correct diagnosis.
• Treatment is given just in time and patient recovers.
• House is rude some more. End credits.

Ok, so where am I going with this? Well I’m certainly not proposing we all adopt House’s method of ‘working’ but I do like the concept of the dedicated Diagnostics Team. This team is highly skilled, highly intelligent and can be called into action whenever the need arises.

Would it be beneficial for an organisation to support a similar type of team as part of a data quality programme? A small team of highly skilled individuals to act as DQ subject matter experts and act as a shared resource across the organisation. Certainly I can see obvious benefits to a relatively immature (in DQ terms) organisation having a team of subject matter experts ‘on call’ to assist with data related projects. A small core team is cheaper to run and will have more immediate impact than trying to spread skilled resource across many business areas. It gives the organisation something to build on and allows them to focus effort where and when it is required.

Of course there are pitfalls and occasions where this just can’t work practically but surely it is still something worth considering? I just recommend that you employ someone slightly more approachable/manageable/polite than Dr. Gregory House.

• Identify risks and associated impacts. Executives live in fear of risks that they haven’t considered and mitigated. Point out the data related risks and explain how to handle them. Be sure to have facts and real examples to back up your points. Use previous data related disasters to your advantage.
• Sell the benefits. Senior management are the key to the success of any initiative and will always want to know where the benefits to the organisation are. Make sure that benefits will out-weigh any associated costs. Try to focus on points related to the reduction of risks and opportunities for revenue increase. Remember that better data = better, more informed decision-making. Use regulatory imperatives and known best practices as leveraging tools when required.
• Don’t treat DQ initiatives as technology initiatives. Data is a corporate asset and needs to be considered as such. Whilst IT should always be involved, it is vital that initiatives are business led.
• Make more data available. The more data available to people and the more people using the data; the more opportunity to uncover problems and increase the demand for better quality data.
• Improve communication and ensure transparency. Have a consistent message and make sure it’s the correct message. Make sure the message is written in business speak. Give people a forum to talk about data and allow them to open up about problems. Ensure there is a non-aggressive and no blame environment even if that means anonymity. Don’t miss any opportunity to spread the word and make findings widely available if appropriate.
• Top-down and bottom-up. Whilst it is important to engage with senior colleagues, it is equally important to identify those people at the coal face. The people who know the data; who use the data; who create the data. Identify and formally recognise existing good practices. Begin to educate and train people at all levels. Promote collaboration to help amass support and create a ‘bigger voice’ to help with senior management engagement.
• Start small but think big. It will always be easier to get smaller initiatives kicked off but ensure they are consistent with the ‘bigger picture’. Make it clear that data quality is not a one-off exercise; it requires a culture change. Organisations need to increase focus on using data and not just producing data.

Obviously there are many, many, more points to make but hopefully I’ve covered a lot of the main ones. If you’ve got 5 minutes I’d recommend heading over to the DataQualityPro.com forum to have a read of the latest replies or indeed feel free to leave a comment here.