News & views

Stop trying to tame the data beast and learn to herd data critters

“The world’s most valuable resource is no longer oil, but data.” – The Economist, 2017  

“Data is crucial to the way government delivers services for citizens, improves its own systems and processes, and makes decisions.” – NAO, 2019 

 

The pressure is on for many organisations to be ‘seen to be using big data’. Designing and implementing big data programmes is a unique challenge.  

It is not about technology  

The difference now is that the challenge has evolved from gripes around just technology and integration (hands up if you’ve built a successful Hadoop data ‘lake’) to also encompass people and ethics. We are told in the media that business decisions are increasingly more data-driven, but our ability to locate increasingly smaller needles in increasingly bigger haystacks is still limited by the abilities of employees as analysts to leverage information, and them being able to navigate the growing importance for responsible data handling and access. Companies have realised that metadata, lineage and data governance help address this; but have also realised that (despite the promises of AI and data science) making effective and appropriate use of big data is still quite a lot of hard work, and costly. Speed of interpretation and inference are everything; the job of the analyst and business decision makers to deliver insight using big data solutions is only getting harder. 

There is a need to grow in-house core data capabilities 

What does this mean for enterprise? At DMW, we are seeing a growing trend in clients wanting to take ownership back of analytics, data capabilities and data stores from outsourced suppliers and System Integrators, in order to bring their staff closer to information, understand its value, and improve capability in analytical decision making. This coupled with convergence of data standards, improved ease of integration and adoption of open source technologies through cloud providers means businesses are more open to self-sufficiency … but struggle with understanding how to achieve it. 

Empowering your people to try and test ideas 

With maturing of cloud, ephemeral platforms, automation and agile and DevOps practice, we believe that the industry is at a tipping point, and learning to herd your data critters, and accepting that it is pointless (and impossible) to tame the data beast, is key. The days of running a costly big data and analytics estate are gone. To succeed in data, businesses must empower their staff to break down the data sources and the problem; and leverage technology that no longer needs to differentiate data as ‘big’. This means changing skills, knowledge and ways of working, and investing in communities and a learning organisation in data and analytics practice that don’t just straddle business and technology; but blur their ongoing interaction. Orienting around blended data and analytics expertise and multi-disciplinary teams responsible for not just delivering, but qualifying value, maintaining delivery standards and contributing back to communities allows for the shepherding of knowledge. Unicorns don’t exist, but by enabling staff to test ideas and leverage solutions they can grow sustainably themselves, you can produce plenty of horses. 

DMW has significant experience in delivering successful big data projects within the public and private sectors and enabling analytics capability.  

If you want to find out more about our approach to data, please read more on our website here or drop us a note to find out how we can help you. 

charts-data-desk-669615

Mark Atterbury

Managing Consultant

Articles

DMW's family day & National STEM/STEAM day 2019

On the 8th November every year, we celebrate National STEM day. But what is it, why is it important and how does it relate to a large ...

Read more

Microsoft Ignite 2019: Satya Nadella's keynote speech

DMW are at Microsoft Ignite this week, one of the world’s leading cloud conferences.

Rob Merrett attended Microsoft CEO Satya Nadella’s ...

Read more

How to manage multiple financial regulations with sustainable technology

With increasing financial regulatory demands ramping up technology & process requirements, there is a growing need for both financial ...

Read more

Raising mental health awareness in the workplace

We asked one of our Mental Health First Aiders, John Kendrick, to write about the importance of good mental health as part of World ...

Read more

SFTR reporting – identifying key challenges and how to solve them

The first deadline for the implementation of the Securities Financing Transaction Regulation (SFTR) is April 2020. In this article we ...

Read more

RegTech and Legacy - the path to better adoption

There are many benefits of using RegTech for the financial services industry, but there are also barriers that need to be overcome in ...

Read more

Market Abuse and Conduct Surveillance: Are you drowning in a sea of alerts?

A multi-faceted approach to transformation is needed in Financial institutions to combat market abuse.

Read more

Surveillance: Readying for the Seismic Shift

To date the overriding priority of surveillance in Financial Institutions has been one of coverage, but a seismic shift is upon us.

Read more

Why celebrating Pride is more than just a banner at DMW

June is international Pride month, where we celebrate LGBT (Lesbian, Gay, Bisexual and Transgender) identities and highlight the work ...

Read more

Disrupt the digital challengers with transformation mindset

Transformation can no longer be seen as a one-off project, and for organisations to survive, it is becoming the new norm.

Read more

DMW on G-Cloud Framework

DMW delivers complex cloud migrations to public and private sector clients, supplying public sector organisations via the G-Cloud 11 ...

Read more

DMW welcomes... Naya Giannakopoulou

Happy International Women's Day! Over the course of recent months we have been delighted to welcome more brilliant women into the DMW ...

Read more

Helping our customers harness the power of cloud

We’re delighted to boost our cloud computing credentials with the announcement that we are now a Standard Consulting Partner in the AWS ...

Read more

DMW welcomes... Hani Qadiri

As DMW continues to grow at a blistering pace, we are welcoming more DMWers through the door than ever. Here is the first in a series of ...

Read more

The Cloud Brick Wall, Part 3 of 3

In the first two blog articles of our series from Konrad Petrusewicz, we started by covering the fundamental challenge of building a ...

Read more

CTO hot take from AWS re:Invent 2018

As the DMW Group team jet back from AWS re:invent, Amazon Web Services major cloud technology event in the states, we offer their ...

Read more

The Cloud Brick Wall, Part 2 of 3

An essential component of any Cloud migration is the actual code base that needs to be run and understanding how it operates within the ...

Read more

AWS re:Invent 2018 Keynote Hot Take

DMW are at AWS re:Invent 2018, the leading cloud technology event from Amazon Web Services in Las Vegas this week. Here’s our take on ...

Read more

The Cloud Brick Wall, Part 1 of 3

The Cloud Brick Wall is approaching. In this 3-part series of blog articles Konrad Petrusewicz maps out how to break through the brick ...

Read more