Ecommerce > Marketing > Operations > Finance >

Data Strategy > Data Science & AI > Data Platforms > Data Visualisation > Data People >

The 5 tools a data analyst can’t do without

Posted on 12 September 20194 January 2023 by Yiannis Pitsillides

The 5 tools a data analyst can’t do without

In this post, we will be discussing the 5 tools that an analyst needs to know to succeed in the field. These tools will help any analyst be faster and better when it comes to analysing data, problem solving and providing insights.

Excel

Across all industries, Excel is the tool that sits on the Iron Throne as the most popular amongst analysts and decision makers. It’s so widely used because:

Everyone has Excel on their machine
It provides great functionality for non-technical people
Data can be manipulated to suit the needs of the user
It’s generally simple to learn and use

So, data analysts will need to be masters of Excel as they are likely to be using it daily. Some practical examples are:

Running / Refreshing / Creating new reports
Where you need additional data for your analysis and the data is an Excel file within the Marketing team – > You will need to get that file and input it in your Database
You need to send out a quick ad hoc request that shows the sales per day – > you can quickly summarise data using Excel
You want to create a lookup from scratch (product or store lookup for example) -> you can create it in Excel and then input it in your database

Therefore, I suggest you make Excel your best friend if you want to pursue a career in analytics.

SQL (or other databases)

SQL is the place where your data will likely be stored so you will need to know how to use it if you want to work with data. SQL (Structured Query Language) is a programming language which data analysts use to interact with their data. Usually, there is a database administrator who is responsible for the maintenance of the Database; making sure the new data is correctly stored in tables and all updates happen flawlessly. If you join a smaller organisation you might have to do this yourself (heads up!); don’t worry though, it’s an opportunity for you to understand how databases work in general.

Data analysts will be using SQL for the following reasons:

Data input – all the relational raw data will be inputted into SQL
Data cleaning – this involves removing empty columns/rows, removing duplicate values, updating data, creating error checks, triggers & deleting unnecessary data
Data transformation – this involves bringing multiple different data sources together or pivoting the data in different ways
Data modelling – This involves creating rules for your data or creating calculated fields
Data storing – saving the output of your analysis or the raw data into tables
Creating views – these views will be a set of queries that bring the data together, transform it and model it for a specific problem (sales Vs target for example). Then you can connect these views with your visualizations tool or Excel and create an automated dashboard

There are more things you can do with SQL, but the above list has the basics that an analyst will need. SQL will also help you in the future when the data will be saved in the cloud (big data tools) because it integrates very well with the major cloud services such as Azure, AWS and GCP.

Visualisations tools (Power BI, Tableau and Qlik Sense)

Visualisation tools such as those above will be relatively new to most businesses, but they are starting to gain traction. As an analyst, these tools will help you communicate your results in a more modern and graphically rich environment verses Excel. Additionally, these tools provide more functionality to Excel such as:

More table / graph options to use
Are more interactive and interconnected
Offer more advanced analytics by integrating with Python & R
Have the ability to handle big data better

The biggest mistake I see when using these tools is that they are mistakenly taken for databases because they have an option to create a ‘data model’ with multiple connections (joins) and calculated fields. If you have medium to big data and try to do the above, your visualisation tool will be slow and clunky – the result being people reverting to Excel. Best practise is to build your model in SQL or a Big Data database and then bring 1 table/view into your visualisation tool – yes, it will take more space, but the report will be fast and usable.

Python and/or R

Python and R will help you when it comes to machine learning (ML) projects such as making predictions, forecasts and run rates. Usually, these projects are run by more advanced data analyst or data scientists.

Personally, I believe that the urge to learn ML will come having mastered the basics mentioned above. In order to start learning, I recommend:

Setting up your Python or R environment (it’s free) using Anaconda and Jupyter Notebooks
Start by running simple statistics like Person Correlations – PLEASE try to interpret the results
Then try running basic unsupervised models like Linear Regression, Logistic Regression, Naïve Bays and Decision trees which are easy to understand
Then learn some unsupervised models like K Means or Hierarchical Clustering
If you master the above, move to Deep Learning models and start using TensorFlow, Keras or PyTorch in your GPU or Virtual Machines.

Fun right? The last bullet point is maybe a bit advanced for an analyst but nothing that cannot be achieved with practical practise (and lots of googling). And you’ve always got the data analyst/scientist community who are amazing at sharing knowledge and always happy to help.

Please note that the ability to learn new software fast is critical as it might be Python and R today but could quickly be something else tomorrow.

Big Data tools like Microsoft Azure, Amazon Web Services (AWS) and Google Cloud Platform (GCP)

Traditional on-premise databases cannot handle the ‘big data’ organisations now generate, thus the need to move to cloud-based solutions like Azure, AWS and GCP. You will still be using SQL, Python and R as the big data tools as they integrate very well together. Typical challenges that exist:

Your organisation has a data lake where all the data is stored, and you need to know how to get there or how to connect to it. Or maybe you want to store data there yourself.
Maybe you want to create a cluster (virtual machine) to run a big data query or model. You will need to know how to configure the cluster yourself which is relatively straightforward; select the processing power you need & the RAM and you are good to go.

The point is that, as we move to the world of big data, there will be new ways of doing things that as an analyst you should be the first to know how to use and apply. Plus, these big data cloud skills are very high in demand!

I hope you enjoyed this read! If you did, then please consider subscribing to my YouTube channel as I will be sharing more content on data analytics. If you have different views or there is something you want to comment on or even just ask me a question, please let me know in the comments section.

https://youtu.be/IO6EtMkim_4

What is data analytics? 5 things you need to know

Posted on 19 August 201913 June 2023 by Yiannis Pitsillides

What is data analytics? 5 things you need to know.

It’s the 21 century and everyone is talking about the big data explosion and that data is the new oil. A report by IDC Data Age (2025) estimated that there will be 175 ZB (we are currently at 40ZB) of data generated by 2025. But this data needs to be stored correctly and analysed.

So, in step data analytics, but what is it – in this post we will be reviewing the 5 things you need to know about data analytics.

What is data analytics

Data analytics is the art of taking some raw unstructured data and using it to build models that leads to better decision making. And in conducting the above process, you will be using a set of tools that will help you be more efficient and effective.

For example – the processes of taking 2+ raw data files in Excel, joining them together, cleaning & transforming them, modelling them and then creating some sort of outcome is the art of ‘data analytics’.

How can data analytics add value?

In general, data analytics helps organisations or individuals make better, informed decisions by justifying/supporting them with data/evidence. Some examples of how data analytics can add value are below:

It can help organisations ‘gain visibility’ across all aspects of their business when measured against KPI’s. Examples include: WoW revenue, footfall, traffic & stock analysis
It can help organisations ‘increase their revenue’ by identifying potential opportunities or underperforming activities
It can improve ‘operational efficiency’ by accelerating all of the tasks, automating them and minimizing manual work
It can ‘optimise marketing campaigns’ by tracking the campaigns, the money spent and the connection between the two
It can ‘increase response times’ with customers, clients and partners
It can ‘identify new trends, new opportunities and new markets’
It can provide ‘real time analytics’
It can provide ‘competitive edge’ over rivals
It can assist in ‘future planning’ by forecasting/predicting the performance of the business

By reading the benefits above, you can see why all organisations realise they need to be ‘data driven’.

Data analytics methodologies (EDA vs CDA)

Data analytics could be separated into 2 ‘methodologies’; Exploratory Data Analysis (EDA) and Confirmatory Data Analysis (CDA). EDA is the process of trying to identify patterns and relationships whereas CDA is the process of using statistical methods to determine whether a hypothesis is true or false.

The kind of tasks performed within EDA are as follows:

Aggregating the data in different ways to get insights (SUM, MIN, MAX, AVERAGE, COUNT)
Visualising the data in different ways to identify patterns (bar charts, line graphs, pie charts, etc).
Checking the distribution of the data
Checking for duplicate values, missing values or incomplete datasets
Filtering the data across multiple categories and investigating if there are any patterns

Think of EDA as almost like the work of a detective where you are investigating a case (dataset) across every possible angle to get insights. All recurring daily/weekly/monthly reports could be considered as EDA analysis as well as the initial stage of machine learning projects.

CDA is where you will test if your hypothesis formed from EDA is true or false. CDA utilises statistical methods such as significance, confidence and inference to challenge any assumptions made in your EDA.

Some of the techniques/models that CDA relies upon are:

Supervised learning models like: Linear Regression, Logistic Regression, Naive Bayes, Decisions Trees, Support Vector Machines & KNN.
Unsupervised learning models like: K-Means & Hierarchical Clustering (although these models are very challenging to evaluate performance of)
Variation Analysis (ANOVA)

CDA is mostly used when you are trying to predict something and you need historic data to create a model that predicts the future.

Quantitate vs qualitative analytics

Data analysis can be broken down into quantitative and qualitative analysis.

Quantitative involves using numbers and quantifiable variables that can be measured statistically or compared to each other. Qualitative analysis involves using text, audio, images, interviews and video to understand the concept of non-numerical data and the story within it.

From my experience, even if you have to do some qualitative analysis, you will need to find a way to categorise your data and make it comparable with something in order to arrive at any conclusions. There are models that can analyse text, images, voice and video and categorise them in a way that you can apply quantitative analytics to them.

Types of Analytics

Another way of breaking down data analytics is by type – there are 4:

Descriptive Analytics: When you use data to describe what has happened over a specific period of time such as the automated daily/weekly/monthly reports that an analyst will typically run – they answer questions like:
- What is the WoW revenue performance?
- Where are we against our targets?
- Which products are performing the best?
Diagnostic Analytics: This step is using ‘descriptive analytics’ to explain why something has happened. For example, after answering that WoW revenue is up by 20%, you will have to investigate the ‘why’ and the what product, category, area, industry, manage, seller has driven that 20% increase.
Predictive Analytics: In this step, you will have to use historic data to make predictions for the future. You can choose statistical methods or rule-based models to make predictions. And you can answer questions like:

How much revenue are we estimated to achieve by the end of the year
Will we meet our targets by the end of the year? By how much? What is the forecast?
What segment does this customer fall in based on his/her characteristics?
What is the opportunity value if we increase marketing spend by 5%?

Prescriptive Analytics: This is the step where you will be making your recommendations. For example, if you have identified that you will not meet the targets by the end of the fiscal year, you will need to recommend a course of action to the business as to how they could achieve those targets.

I hope you enjoyed this read and have gained a solid understanding of what data analytics is and what it entails! If you enjoyed this read, then please consider subscribing to my YouTube channel as I will be sharing more content on the subject of data analytics. If you have different views or there is something you want to comment on or ask me a question about, please let me know in the comments section.

https://youtu.be/nD3410mCAhA

References:

Reinsel, D., Gantz, J. and Rydning, J. (2018). The Digitization of the World – From Edge to core. [ebook]. Available at:

https://www.seagate.com/files/www-content/our-story/trends/files/idc-seagate-dataage-whitepaper.pdf [Accessed 18 Aug. 2019].

The skills you’ll need to be a great data analyst

Posted on 10 July 20194 January 2023 by Yiannis Pitsillides

The skills you’ll need to be a great data analyst

The world is exploding with data; more data that we can possibly handle. Organisations are seeking to employ more data analysts. Young, talented and ambitious individuals are keen to start a career in data. But what does it take? In this post, we will be discussing the 5 + 1 skills that an analyst needs to be successful in this field.

Ability to turn data into insights

The most important skill that a data analyst (even a data scientist) must learn is the ability to turn data into insights. This is often overlooked, but the reality is, that if you are not able to translate your analysis into valuable and actionable insights then your organisation will not value you. You will need to analyse your data by measuring the right thing and asking the right questions.

To really master insights, you’ll need to be able to tell a story using the data. You’ll need to articulate the why and present your analysis in the right order with very clear visuals.

Problem Solving Skills

In the real world of data, no problem is the same and no solution is the same either. There are no fixed rules that mean you can follow and solve every problem simply.

You will need to be able to solve problems efficiently and effectively. By efficiently I mean that we do not live in a perfect world where we have all the resources available to us; we need to solve the problem with what we have and ensure it’s the best possible solution we can deliver. By effectively I mean that our solution should solve all of the problem and not just some of it.

Analysts should clearly state the problem they are trying to solve. They then need to identify the information they need to solve that problem, gather that information and join it together, they can then use their well-honed analytical techniques to model the data and start analysing the outcomes that arrive at a solution and so answer ‘why’ this has happened and ‘how’ we can tackle it – problem solving – done!

Technical Skills

‘Technical skills’ are the most obvious skills a data analyst will need. Technical skills will help the analyst to be faster and better when it comes to problem solving, analysing data and providing insights. At an industry level, the most used software is Excel so whether we like it or not, data analysts should be masters of excel. Managers, Senior Managers, Directors and Executives will want a relatively simple software package they understand and that in some cases can give them the ability to manipulate the numbers themselves.

The next most popular technical skill is SQL; or other databases where you can still query using SQL. Looking at the fundamentals, all the data that you will be using would sit within SQL, so it makes sense to learn SQL in order to extract that data and start your analysis. SQL also integrates with the majority of visualisation tools (and big data tools like Azure, AWS, GCP) allowing you to automate all your reports.

I would suggest learning how to use visualisation tools like Power BI, Tableau & Qlik Sense. These visualisation tools have more functionality than Excel and are able to handle big data better than Excel.

Worth noting that if you use there tools as ‘databases’ and have your ‘modelling’ and ‘joins’ within the tool then the tool will be effectively useless as it will take ages to run/filter. I have personally experienced this, many times in my day job and it’s one of the main reasons people find those tools hard to use – this can often lead to a reversion to Excel.

Finally, I would suggest learning Python and/or R. They are really useful skills when it comes to machine learning (ML) projects such as making predictions, forecasts, run rates and other statistical based projects. Usually, these projects are run by more experienced data analysts or data scientists.

Please note that the ‘ability to learn new software fast’ is critical as it might be Python and R today but that could change tomorrow. As an example, digital data analytics is relatively new and has new tools and processes to measure the data.

Communication Skills

Data analysts don’t just interact with computers, (although some of us wish that were the case) they also interact with people often for the following reasons:

You might have done a great piece of analysis and have to present your results to the business
You might need some data that is not in the database and you will have to go and talk to other people to get that data
You might have been asked to help a certain team analyse their performance and you’ll need to sit down with them, understand their business and map the project/analysis from conception to conclusion.

So yes you are going to need to have good communication skills too.

Critical thinking

Critical Thinking is not the same as the problem solving described above. Critical thinking is when you challenge your own presuppositions/assumptions.

Questions like:

Maybe I am wrong and not thinking this through clearly?
Am I doing this right?
Is there another way of doing it better? Faster?
Is there a hidden reason why ‘x’ is asking me to follow this direction?
Why this way over another?

Problem solving will typically have an end date as problem while critical thinking is an on-going process that you should apply throughout your career. The reason I have included both is because as an analyst, critical thinking will help support problem solving and should mean your recommendations are stronger.

5 + 1. Creativity

Creativity is an art however and wherever you apply it. As a data analyst, you will be presented with multiple scenarios that often have no right or wrong answer. Hence, you will need to get creative when providing solutions. Creativity can be applied in the following scenarios:

You can get creative on the type of model or software you use for your analysis
You can get creative when designing new reports / visualisations for the business
You can get creative in including more data sources in your analysis to get a more complete picture
You can get creative on how to present your insights by telling a story
You can get creative by experimenting how to automate all your work, so making it faster

https://youtu.be/I1MOn8kyc14

Getting Started as a Data Analyst

Posted on 19 June 20191 May 2024 by Yiannis Pitsillides

Getting started as a data analyst

In very simple terms, a data analyst job is to translate numbers into plain English so the business can gain a better understanding of performance. However, in terms of their actual day to day routine, it’s a bit more complex than that. In this article, we will be discussing the 5 + 1 things that a data analyst does in his or her day job and the skills required for each step to be successful. Additionally, we will touch on the kind of tools and data, analysts use in each step.

1. Reporting

As a data analyst, you will be spending a lot of your time either creating, refreshing or running reports. These reports will likely be in the form of Excel or PowerPoint with commentary, and use visualisation tools like Power BI, Tableau & Qlik Sense. If you are creating a new report, then you will probably be using SQL, Excel or any other database; as it’s the place where the raw data will be saved after you collect it from other people or data sources. When I started working as an analyst, I was given responsibility for 5 different reports. I had to make sure that all the reports were refreshed on time, with the correct data and that they were sent out to the right stakeholders at the right time. For example, I had a marketing report, a retail report, a product report, etc and each report had a different group of stakeholders. This is actually a good way to start your career as you get exposure to the wider businesses and start to understand how different teams use their reports. This will give you the confidence to start creating your own reports. The kind of skills you need in this initial step are the following:

data gathering skills
data cleaning skills
data transformation skills
data storing skills
technical Skills – SQL, Excel, database knowledge, BI/Visualisation
organisational skills & attention to detail
punctuality – ensuring the reports are out on time

2. Analysing the data

After refreshing the reports, you will be spending time looking at the data and trying to understand it, looking at the patterns and the performance of what you are reporting. In some cases, the reports alone will not be sufficient for your analysis and you will have to spend extra time pulling and analysing more data from the database in order to complete your analysis. After the analysis, you will be writing your commentary/insights – think of this as a story of your findings. While going through the process described above you will then to need to check you work for any obvious errors/mistakes in the data/reports, having business knowledge is critical here. The kind of skills you need in this step are the following:

problem solving – no solution will be the same hence your problem-solving skills will be challenged
ability to ask the right questions
business knowledge
fixing mistakes

3. Presentation of results

After finishing your analysis, you will have to present the results back to the business. This will usually be in the form of a PowerPoint presentation. This is often the skill that the business values the most in a data analyst. It does not matter how good your coding/ modelling/predicting skills are if you are not able to turn your findings into insights and business recommendations. By mastering the ability to turn data into insights and being able to communicate this to the business will see your career start to move in a positive direction. The kind of skills you need in this step are the following:

insights
storytelling
communication skills
agile thinking
strategic thinking – align your analysis with the business’s strategic objectives

4. Ad hoc analysis

The next thing you will likely be doing is ad hoc analysis. Ad hoc is usually a specific request that crops up on the back of your initial presentation or frankly any meeting with stakeholders in general. It’s typically just a one-off analysis on a subject of interest. For example, you have identified from your analysis that category X had an amazing week last week. Maybe your manager will ask you to do an ad hoc analysis on why category X performed so well? What are the drivers? Ad-hoc analysis is a good way to show your creative thinking and problem-solving skills as it’s different from the standardised reports that the business has. In this step, you can also demonstrate your machine learning (ML) skills (if you have some); that is if there are applications of ML that would benefit the problem you are trying to solve verses what can happen which is trying to force ML in because you think it will impress. The kind of skills you need in this step are the following:

problem solving
data gathering
asking the right questions
time management

5. Automating all processes

As an analyst you will have to learn to work smart and not just hard. The majority, if not all reporting should be automated using SQL (or similar) to Excel or SQL to one of the visualisation tools. There is a smart way of creating reports so that as soon as there is new data in the database, all you have to do is press “run” and “refresh” and within 10 secs all the data is cleaned, transformed, modelled and refreshed. The obvious benefit of this, is that you can save a significant amount of time which you can then spend analysing the reports and adding more value to the business. The kind of skills you need in this first step are the following:

technical skills
methodical thinking – you’ll need to think hard about the order of doing each task and when and why.
ability to see the bigger picture – if you recognise the benefits of automation then you can really add value to the business.

6. Build machine learning models

This step is mostly for advanced data analysts that know how to use machine learning. It’s not something that an analyst will spend much time doing but at some point in your career, you will build or be involved in building a machine learning model. Personally, I believe that the urge to learn will come naturally in all good data analysts after mastering the basics discussed above. The kind of skills you need in this first step are the following:

ML knowledge – start by learning some basic models like linear regression, logistic regression, naïve Bayes & decision trees for supervised learning & k-means for unsupervised learning.
Python or R skills – I recommend Python but R is equally good.
statistics – correlation analysis will be very useful
advanced ML libraries like TensorFlow, Keras and PyTorch (GPUs or VMs)

Author: Yiannis Pitsillides