STUDY OBJECTIVES
This chapter will help you gain an understanding of the following concepts:
The need for data collection and storage
Methods of storing data and the interrelationship between storage and processing
The differences between batch processing and real-time processing
The importance of databases and the historical progression from flat-file databases to relational databases
The need for normalization of data in a relational database
Data warehouse and the use of a data warehouse to analyze data
The use of OLAP and data mining as analysis tools
Distributed databases and advantages of the use of distributed data
Cloud-based databases
Controls for data and databases
Ethical issues related to data collection and storage, and their use in IT systems
Think about the volume of sales transactions that occur on the websites of large Internet retailers such as L.L. Bean, Lands' End, and J.Crew. These companies each process an average of approximately 120,000 transactions each day on their websites. For each of these transactions, important data must be collected about the customer, location, payment, and the items sold.
Even more overwhelming is the volume of sales transactions that are processed by Wal-Mart on any given day. In addition to its Web-based sales, consider Walmart's thousands of retail centers with several check-out lines at each location and long hours of operation. Think about the number of accountants and computers that might be required to manage all of the related records. It is no wonder that Walmart has one of the largest databases of any business organization in the world.
The Walmart database continually grows with new transactions. Some estimate that Walmart adds 1 billion rows of data per day. In addition to the size of the database, it is also growing faster. The company attaches RFID chips to merchandise so that inventory purchases, movement to stores, and sales are tracked in real time. Since the data for these events get added to the database so quickly, the database grows and becomes more useful for immediate analysis. This allows Walmart to more quickly analyze and forecast inventory needs.