Guest blog post by Michael Walker
More and more frequently we see o rganizations make the mistake of mixing and confusing team roles on a data science or "big data" project - resulting in over-allocation of responsibilities assigned to data scientists. For example, data scientists are often tasked with the role of data engineer leading to a misallocation of human capital. Here the data scientist wastes precious time and energy finding, organizing, cleaning, sorting and moving data. The solution is adding data engineers, among others, to the data science team.
Data scientists should be spending time and brainpower on applying data science and analytic results to critical business issues - helping an organization turn data into information - information into knowledge and insights - and valuable, actionable insights into better decision making and game changing strategies.
Data engineers are the designers, builders and managers of the information or "big data" infrastructure. They develop the architecture that helps analyze and process data in the way the organization needs it. And they make sure those systems are performing smoothly.
Data science is a team sport . There are many different team roles, including:
Data change agents.
Moreover, data scientists and data engineers are part of a bigger organizational team including business and IT leaders, middle management and front-line employees. The goal is to leverage both internal and external data - as well as structured and unstructured data - to gain competitive advantage and make better decisions. To reach this goal an organization needs to form a data science team with clear roles.