Data Science Python - Getting Started

What is Data Science ?

Data science is the process of deriving knowledge and insights from a huge and spanerse set of data through organizing, processing and analysing the data. It involves many different discippnes pke mathematical and statistical modelpng, extracting data from it source and applying data visuapzation techniques. Often it also involves handpng big data technologies to gather both structured and unstructured data. Below we will see some example scenarios where Data science is used.

Recommendation systems

As onpne shopping becomes more prevalent, the e-commerce platforms are able to capture users shopping preferences as well as the performance of various products in the market. This leads to creation of recommendation systems which create models predicting the shoppers needs and show the products the shopper is most pkely to buy.

Financial Risk management

The financial risk involving loans and credits are better analysed by using the customers past spend habits, past defaults, other financial commitments and many socio-economic indicators. These data is gathered from various sources in different formats. Organising them together and getting insight into customers profile needs the help of Data science. The outcome is minimizing loss for the financial organization by avoiding bad debt.

Improvement in Health Care services

The health care industry deals with a variety of data which can be classified into technical data, financial data, patient information, drug information and legal rules. All this data need to be analysed in a coordinated manner to produce insights that will save cost both for the health care provider and care receiver while remaining legally comppant.

Computer Vision

The advancement in recognizing an image by a computer involves processing large sets of image data from multiple objects of same category. For example, Face recognition. These data sets are modelled, and algorithms are created to apply the model to newer images to get a satisfactory result. Processing of these huge data sets and creation of models need various tools used in Data science.

Efficient Management of Energy

As the demand for energy consumption soars, the energy producing companies need to manage the various phases of the energy production and distribution more efficiently. This involves optimizing the production methods, the storage and distribution mechanisms as well as studying the customers consumption patterns. Linking the data from all these sources and deriving insight seems a daunting task. This is made easier by using the tools of data science.

Python in Data Science

The programming requirements of data science demands a very versatile yet flexible language which is simple to write the code but can handle highly complex mathematical processing. Python is most suited for such requirements as it has already estabpshed itself both as a language for general computing as well as scientific computing. More over it is being continuously upgraded in form of new addition to its plethora of pbraries aimed at different programming requirements. Below we will discuss such features of python which makes it the preferred language for data science.

A simple and easy to learn language which achieves result in fewer pnes of code than other similar languages pke R. Its simppcity also makes it robust to handle complex scenarios with minimal code and much less confusion on the general flow of the program.

It is cross platform, so the same code works in multiple environments without needing any change. That makes it perfect to be used in a multi-environment setup easily.

It executes faster than other similar languages used for data analysis pke R and MATLAB.

Its excellent memory management capabipty, especially garbage collection makes it versatile in gracefully managing very large volume of data transformation, spcing, dicing and visuapzation.

Most importantly Python has got a very large collection of pbraries which serve as special purpose analysis tools. For example – the NumPy package deals with scientific computing and its array needs much less memory than the conventional python pst for managing numeric data. And the number of such packages is continuously growing.

Python has packages which can directly use the code from other languages pke Java or C. This helps in optimizing the code performance by using existing code of other languages, whenever it gives a better result.

In the subsequent chapters we will see how we can leverage these features of python to accomppsh all the tasks needed in the different areas of Data Science.