zoom mysterious world

What is Data Science ?

Data Science means on the basis of analyzing the data we take a marketing decision. Moreover we take analyzed data on behalf of which we took a decision. This is called as Data Science. In this way ,you can make this as an programming excel or use in any other platform.

Data Science is an inter-disciplinary field that uses scientific methods, process, algorithms and systems to extract knowledge and insights from many structural and unstructured data.

Concept of Data Science:

In Data Science we hear a lot about Artificial Intelligence and Machine Learning , which are going to change the world and how Internet of things will make everyone’s life easier. But what is the one thing that underpins of all these revolutionary technologies? The answer is Data from social media to IOT devices for generating inmeasurable amount of Data . Considering the cab service provider like Uber. As per current era we all must have used Uber. What you think about Uber is a multi-million dollar worth company? Is it that availability of cabs or Is it their service ? Well the answer is Data which makes them very rich, but wait Is Data enough to grow a business, obviously it isn’t we must know how to use Data to draw useful insights and solve problems. This is where Data Science comes in. In simple words Data Science is the process of using Data to find solutions or to predict outcomes for a statement with a problem better to understand Data Science. Data Science is related to Data Mining and Big Data.

Now in Data Science as we discussed earlier two things are more popular one is Artificial Intelligence and the second one is Machine Learning. Let us try to understand these concepts in brief.

You own a small company and people may have an idea to make CRM , so what will you do is you may hire someone  and tell him to collect Data on a regular basis. Basically while growing , we need to hack at least 5K – 6K Database to analyze it properly. Once you collect the number of account then you can easily make it on excel and the work will be done. But if you have a lot of Data ,  For e.g., you are trying to analyze Google’s  Data or trying to analyze the Data of that company which has over 100 branches and the daily Data of each branch is more than 1lakh. So Are we able to make it on excel sheet ? Assume that there will be many things which would work on monotonous basis. Such as first addition will be done then subtraction and the process will be going on. So here we do the same thing in programming instead of excel so it will be called as Data Scientist.

What is Machine Learning?

It means the machines are trying to learn how people behave. For e.g., I just said “Ok Google” so in many phones this instruction opened and on some phones it may not, so after that machine realized it didn’t open. Let me give proper explanation with another e.g., Whenever we chat on Whatsapp they ask us for rating. Some people rate 2 stars some 3 or 5 stars and some might be different. So on behalf of the Data they try to analyze the Data on a broader version. There are some various factors on which they try to analyze how was the connection, what was the device specifications such as its RAM, its Processing and many more. On excel we can consider maximum 2 or 3 factors and if we want to consider more than that we have to set up big software , that’s why the thing has been shifted to programming so we don’t need to spend money on complex software while analyzing the Big Data.

Students whoever knows well about statistics , they need to use statistics in order to analyzed bigger Data. Earlier when technology was not updated then statistics were used to analyze Data. This is the reason we are still learning about statistics. But now the software is launched on which we just put the statistics and answer will come up on the screen automatically. But suppose we need to make a thing for which no software is available so in that situation we will make our own code to execute the data of statistics.

Benefits:

Products will improve. People will give ratings to Whatsapp so in this way we can give Data to the machine, and doing business in this way(as discussed in above) ultimately the work will be improved. If there is a service industry as we know about the recent incident if a specific store is getting bad reviews , overall company can see that it is not going good. That’s the reason A big brand always ask you for filling a feedback form after using their services, whereas earlier the feedback form was used to fill manually on papers that’s why company hire people for Data entry. They just record the Data in excel sheet. But where all the Data go for compiling. Suppose there are 30 stores in vizag, in other states there would be one person whose duty is to collect Data of all the stores of Vizag and the same thing happens in the other states. There would be another person who will put all the files of different states together and then try to analyze Data in a broad term. He will try to figure out what people likes or dislikes. Are they comfortable with new products or not. People can’t even imagine how big the thing is. In every process Data Science is being used. It doesn’t matters you are selling a product on Amazon but we are discussing about a big companies but not small ones, only those companies who have big Database.

Data Science are used everywhere like if people want to improve their product service or even if their process. For e.g., you have a machine which manufactures something. So a person would be there who was assigned to check machine, he analyze and reports that where the machine actually lacks and how much time it takes to process and many other things.

A role of Data Scientist is analyzing the Data, and then takes a decision but not coding. Suppose we take a large Data and test it. For e.g., We show 2 types of advertisement to people.

One advertisement is related to Orange candy and other is related to Mango candy. Now there are many people who likes Mango candy. So we come to know people more prefer to Mango candy and after analyzing the Data we increase the manufacturing of Mango candy and decrease the manufacturing of Orange candy. So those are the works of Data Scientist. In beginning things won’t be like but observing overall a discipline forms. In future large scale companies will be working Data Driven approach. Whenever we go to a mall and many a times people offer a free sample of product and only ask us for our feedback form. They take our feedback and simply analyze Data and takes a relevant decision.  

Concept of Big Data:

Big Data means large collections of data. Big data is one of the latest technology . Earlier to this concept , mostly preferred technology is Data,  which is a traditional database.

In 1970, William Ford who designed this concept to overall database and Revolutionary Database management. Here data is stored in a structural form. The data which stored in Revolutionary Database Management system is mostly in structured form i.e, in tables . But in recent days mostly 90% of people prefer for unstructured data like in photos, videos, blogs, etc, where in videos we can find  videos only, in photos we can find photos only so the data will be in unstructured form with a huge varieties. Comparing the Big data with data, the data will be small so it named as Small Data and been said that earlier people use to follow Small Data but now era has been changed so far the technology as well as with the people too and now it’s a trend of Big Data.

What is Big Data ?

Before we learn about Big Data let us learn about what is data. Data is a collection of information. e.g., to know about some topic we collect information such as  if people are reading this blog or this blog related videos means they are collecting information about Big data. In Instagram if people post a picture i.e, also called as data or  guys if you share some images in Facebook that will also be called as data. So anything, digitally happens or digitally written i.e., known as Data. We have number of varieties  of Data. Simple kind of data that we can say as a video. Now lots of volume of Data if we mix i.e.,  called as Big Data. Big Data is nothing but large amount of complex data is simply said as Big Data.

What we do in Big Data?

Huge companies use process in this. My blog is simply a data, whereas the number of data used to be processed and the required information is filtered  for any purpose, so through this process company starts making benefit of this and use it in decision making.

Now we may ask How company uses Big Data in Decision Making ?

 People may heard that facebook leaks its data like in elections , people win through this Facebook website data how is it possible because in Big Data number of peoples Data is existed and through the Facebook we come to know the people’s  interest and sometimes shows that interest could also be changed. So whatever through this website tricks people usually wins elections . Moreover sometimes big companies had an urge of a printer yet the company not purchased any printers but still the printers are shown to the company so why is this happens means recollecting large amount of your company data, the needed data would be shown digitally through this way. In this way Big Data is used and processed.

We may not sample  but simply observe and track what happens. Big Data means a vast collection of Data , by  using this data company incurs the profit and also may use in Decision Making. As the previous years stored data of a  company is been collected and company usually checks which year  profits got maximized  and in which year they incur losses, what will be their decision when company earns profits, so all this based on analyzing which action should they take and which action they should not  , so that a company incurs  maximum profit. So what we can see here a Data how actually helps in Decision Making.

Usually , In Big Data a useful Data which we refine is known as processing and number of techniques we have like Data mining, Data Visualization,  Analysis, Search, Sharing, Querrying, Updating, Information privacy and Data source.

Types of Big Data:

Structured Data:

Structured Data is in a standard form. Everything will visible clearly  e.g., under name column the name would be mentioned, under the age column the age would be mentioned. Its just like in a tabular form. So no need of any hardwork here because  the Data is simply clear.

Unstructured Data:

Here Data is not clear, like we can see videos, audios, images, emails, all these are in unstructured pattern. No particular format is used and from this data we need to be refined. So this type of Data is called Unstructured Data.

Semistructured Data:

Structured Data + Unstructured Data = Semi Structured Data

In this different varieties of data are there the data may be like structured data and also in unstructured form like CSV file.

Characteristics of Big Data:

Big Data originally associated with three key concepts which is also known as 3V’s:

Variety:

In Big Data all kinds of varieties are available its not like any videos are available or any images are available but all types of data is available such as videos, audios, written content, text, Email, PDF, Excel, approximately all varieties of Data is available over here. Big Data represents varieties of Data.

Velocity:

In which rate of speed the Data is being processed like people upload Data or Retrieve Data how fast will be the processing time to take for all these steps.

Volume :

Volume means a large amount of data, means thousands/lakhs GB of Data then it is known as Big Data.

Therefore Big Data often includes Data with sizes that exceed capacity of traditional software to process within an acceptable terms. Current usage of this term i.e,  Big Data tends to refer use of Predictive Analysis or certain other advanced Data Analysis.

Introduction about Devops:

Devops is absolute need in every organization. It is latest buzzword in IT industry and it will remain that way until every organization out of their adopts. Devops or one of the variation of software world.

Devops apart from being Buzz word from internet it is coming from a clip compound words coming from two words.

                             Development + Operations = Devops

Let before we proceed about the problem in the statement Devop is trying to solve every need to understand what Devop is actually “NOT”

What is Devop?

Concept and How Devops work:

Devop is a concept i.e., used in the application Life cycle management and make sure that your development team is absolutely relaxed  and your operational team the team which handles all the servers all those amazing stuff are relaxed are working with the sink each other so that whatever the product or feature you want to give your end user you can do this with absolutely in smooth manner.

Now let us go to little deep that What is a big problem that Devops is trying to resolve:

Whenever the Application is being developed there are variety of phases of its development now when the Application is in small scale maybe designing a website or maybe designing a mobile apps then here is all good and happy because here ultimately manage everything you just design a website and maybe putting down in severs cloud whatever i.e., the features are very next day available the entire world to be used. The things actually changed whenever you work on big scale application say you tube, swiggy, zomato, flipkart, Amazon or may be any other part of these website. When these websites are being done we have variety of teams to mange parts the architecture of the applications putting it onto a midsize company usually there are two teams.

Team A:

Development

In this development team regards all developers such as Write code, Design new feature, Test Feature all these are on development so usually sometimes testers are also included in this team. Now this team designs a product checks out and says absolutely fine.

Team B:

Operations:

On the other team we have operations. The role of operations team is to manage all server configuration and not just only server configuration at couple of other things as well. These teams are responsible for managing such as application phases such as high traffic during the weekends, how they are going to manage then do we need to scale up, do we need to increase RAM over the servers or do we need to integrate any cloud server.

Now Team A Developers Team actually packs up the next update the features they want to roll out and sends it to the operations team.

Usually Developers team thinks that operations team have nothing more work to do and whenever they send such any updates they are going to deploy but what actually happens is operation team also gets busy in managing and checking a number of things including a scaleability, kind of traffics that getting, kinds of security as well. Usually in big applications these deployments happens in once in month or may be two times a month and only two time you can publish any code to the production level in sometimes creates a development team into frustration they did all. This creates a lag user doesn’t cares what’s happening what kind of fighting happens between development and production team.

 User always concentrates on new features as soon as possible or else competitor may release that faster than you. This exact problem can be solved through mindset a philosophy being used in Devops. In Devops the development team and the operation team doesn’t sit into completely different arena. They sit together they discuss side by side and even exchange the roles and responsibilities so that every person knows what happens in the operations and operation team know how teams work in development side as well. This exact mindset and philosophy that everybody knows what is happening in the development and what’s everything happening in operation side is called as Devops.

Infact the logo of the Devops is an Infinite cycle, the application development is infinite process it includes the variety of steps logo of Devops explains Devop pretty perfectly. Apart from this Devops also focus on lot of things into Automation as well. The more things are automated the more things are going to be free.

 Infact interact people and lots of things and tools as well are here to understand. Now if a Person says Devops also have tools like Peppet, Jenkins, code editors etc. Since application development  to be reaching truly to the users these all tools are to be used. Its not about learning any essential tool in all this Devops is all about minset moreover understanding in every phase of development cycle and understanding what happening on the other wall.