- The role of data analytics in accounting and finance
- Big data in accounting
- Big data in finance
- Applications of big data analytics in finance
- Data mining in accounting
- Accounting and data science
- How accounting data analytics benefits accountants
Since their inception, the accounting and finance industries have proven their worth to businesses by delivering new forms of value, whether through higher revenue or more efficient operations. No technology offers more promise for delivering innovative sources of value to businesses than advanced data analytics.
In particular, the use of data analytics in accounting and finance has been a major factor in boosting profitability and reducing the costs of doing business.
- Customer analytics identifies consumer spending habits and other behaviors to spot market trends and anticipate new opportunities.
- Algorithmic trading automates the process of monitoring stock prices. Algorithms place buy and sell orders for when specific conditions are met without requiring direct human intervention.
- Unstructured data that previously wasn’t available for analysis, such as social media feeds, broadens the scope and improves the timeliness of business analytics.
According to a 2020 survey of accounting professionals by software vendor Sage, 44% of accounting firms were using advanced and predictive analytics that leverage big data, or planned to do so in the next 12 months. Among emerging technologies, only 5G had a higher adoption rate among accountants (46%).
The role of data analytics in accounting and finance
Advances in data analytics create opportunities for accountants and finance professionals to offer higher-quality services to their business clients in three areas:
- A broader and deeper perspective on the business’s financial and other operations
- More accurate predictions of future market and industry trends
- Automation of routine tasks to improve accounting accuracy and reduce costs
Data analytics in accounting uses advanced techniques to help firms capitalize on the massive amounts of data they collect. The goal is to create value and growth by leveraging three emerging technologies:
- Computing power and cloud storage have grown tremendously. Datasets can be large and complex because services such as Amazon Web Services offer scalable processing and storage that expands automatically to meet demand.
- Data sources such as internet service providers, social media platforms, mobile apps, government and other open sources, and sensors and other embedded devices are widely available.
- A digital infrastructure now exists that is based primarily on open-source software. Open networks make it easy for data specialists who have expertise in leveraging data to communicate with domain specialists who are experts in specific fields, including accounting and finance.
How analytics is transforming the accounting and finance industries
The accounting and finance fields are being reinvented by new technologies. An international survey of accountants conducted by Sage in 2019 found that 90% of respondents believed there has been a cultural shift in accountancy. The changes are apparent in hiring practices, business services, and the industry’s approach to analytics, artificial intelligence, and other emerging technologies.
This shift embraces the opportunity to expand the range of services accountants and finance professionals offer their clients and improve the quality of the services they currently offer. The changes will affect many areas of accounting and finance:
- The skills and training required to qualify for accounting and finance positions
- Technology adoption and integration with existing processes and procedures
- Greater client expectations about the quality and type of services available
One effect of the cultural shift in accounting and finance is that companies are increasingly recruiting candidates from nontraditional backgrounds, according to the Sage survey. This change is an attempt by accountants to better represent their clients and for accounting firms to add a broader range of skills they can tap to serve their business customers.
- Percentage of accounting firms recruiting candidates from nontraditional backgrounds: 82%
- Percentage of firms seeking candidates who have industry experience outside of accounting: 43%
The accountants surveyed emphasized the importance of preparing the industry for analytics, AI, and other technologies.
- Percentage who believed the accounting industry needs to move faster to adopt new technologies or risk falling behind international competitors: 85%
- Percentage who stated that the primary benefit of technology adoption is increased productivity: 56%
- Percentage who cited time savings as the principal benefit of new technologies in the accounting field: 27%
Risks of analytics: ethics, privacy, and potential for errors and misuse
Any business process that collects customer data must ensure that any use of the data protects the privacy and other rights of those customers. One of the new ethical dilemmas related to AI-based algorithms in particular is the lack of consent when the systems create private data that didn’t previously exist. An example is an algorithm that automatically links a person’s bank account activity with the location tracking and call history collected from the individual’s cell phone.
The Digital Analytics Association has created a code of ethics for web analysts that emphasizes honesty and personal accountability to protect privacy, operate transparently, and educate web users about their work. The analytics service Blast Analytics offers a code of ethics for data analysts that describes eight ethical guidelines for the profession:
- Safeguard customers’ personally identifiable information (PII) rigorously.
- Don’t try to hide bad news in the analytics results. Explain the reasons for the bad news.
- Don’t misrepresent the data to emphasize something that isn’t important or de-emphasize something that is important.
- Don’t attempt to skew analytics results to favor a specific outcome.
- Always protect the trust clients place in analytics by being honest and correcting mistakes as quickly as possible.
- When the quality of data comes into question, let the client know that the results may be unreliable.
- Always add value to the client’s business and give clients full ownership of their own data.
- Follow all data governance rules to the letter and ensure that employees and clients understand the regulations and compliance matters that relate to their customers’ data.
In addition to the eight guidelines for ethical analytics, the code lists three areas that accountants and finance professionals should focus on when applying data analytics in their work:
- Data analytics is a process of testing and iteration to continually experiment with results and then apply the lessons of each test. The freedom to make mistakes early leads to fewer errors in the future.
- Data analytics is intended to have a positive impact on the profitability of business clients of accountants and finance professionals. The process converts data into knowledge that leads to more effective business decision-making.
- Be accountable for the data and transparent about any potential shortcomings or mistakes in the analytics process. Step back from the technical details of the work to view the analytics process and results from the client’s perspective.
Big data in accounting
The goal of big data in accounting is to collect, organize, and tap data from a variety of sources to gain fresh business insights in real time. For example, instead of relying on monthly financial reports for their analyses, accountants and financial analysts have access to up-to-the-minute information from any location with a network connection.
- Their analyses can now include unstructured data, such as audio, video, and images, as well as email and text files, social media posts, website content, and information gleaned from mobile devices. In the past, analysts were limited to analyzing data that could be converted to a structured format, usually a spreadsheet or relational database.
- Data analysis is enhanced by using visualization software that offers accountants and their clients unique views of the data that supports their decisions.
- Auditors are now able to process larger amounts of accounting data in a variety of formats simultaneously, which means their work is done more quickly and is more accurate.
- Big data improves risk analysis by providing accountants with access to more timely data. Advanced analytics tools allow them to process the data quickly.
A survey conducted by the Institute of Management Accountants found that 67% of accounting firms have either implemented big data or plan to do so. Among the 32% of firms that have completed implementation, these are the most popular uses of big data:
- Performance measurement (used by 100% of firms that have implemented big data)
- Strategy formulation (74%)
- Research and development (50%)
- Order fulfillment (25%)
- Rationalizing products and services (25%)
Big data presents opportunities to improve the quality of accounting services in three primary areas:
- Data governance and privacy: The extensive use of personal information requires that accounting firms monitor their compliance with regulations relating to the security and appropriate use of sensitive information in big data applications.
- Gaining business insights: Accountants and other finance professionals will need to work more closely with business managers to understand the processes and functions that they rely on and better support the business decisions that affect those processes.
- Risk management: Business managers need a better understanding of the external forces that impact their operations, including regulations, supply-chain disruptions, and threats to the company’s reputation and brand. They also must be made aware of obstacles to the company’s growth plans and product strategies.
To meet their business clients’ needs, accountants will need to learn new skills relating to how datasets are structured, organized, and applied, as well as tools for conducting strategic analyses and collaborating across functional teams in an organization.
Resources on big data in accounting
- Journal of Accountancy, “Resources for Teaching Data Analytics in Accounting” — Tools for studying the use of data analytics by accountants including software programs, case studies, white papers, and training videos
- Strategic Finance, “The Impact of Big Data on Finance” — An overview of the results from a survey of accountants that examines the most common sources of big data for accounting firms and their strategies for implementing the technology
Big data in finance
The use of big data in finance combines tools that create, capture, manage, and process financial and other information with innovative approaches to convert the data into financial intelligence that guides business decisions. In addition to the size of the data pools that the tools work on, big data has two other characteristics that finance requires:
- Machine learning and other advanced analytics techniques are needed to account for variations within small data samples.
- Natural language processing, speech recognition, and image processing allow the systems to work with unstructured data beyond the capabilities of analyses that are limited to data housed in spreadsheets, databases, and other structured sources.
Big data helps financial institutions address heightened competition and regulation while meeting the rising expectations of their clients by taking advantage of the four V’s of big data:
- Volume refers to the increasing size of the datasets that the financial industry must process and analyze, which now measure in the petabytes (one petabyte equals 1 million gigabytes).
- Variety relates to the many different data sources that big data applications tap to create analyses that more accurately represent a business’s financial operations today and in the future.
- Velocity refers to the high speed at which data is created, which requires distributed processing techniques to collect and curate information in many different formats and contexts.
- Veracity describes the quality of the data being analyzed, especially whether the data is consistent and certain. It also relates to the data’s ready availability and controllability.
The four V’s are sometimes referred to as the five V’s when value is added. Value in this context means that the data contributes in a meaningful way to the analysis rather than being extraneous.
The three primary areas where big data is applied in finance are algorithmic trading, compliance, and data quality:
- Big data is a key to algorithmic trading, which uses a computer program to execute financial trades much faster than human traders can. The algorithm uses mathematical models to define the parameters and instructions that control the trade decision, including timing, price, and quantity.
- Big data helps financial firms confirm they comply with all government regulations, particularly those relating to data security and privacy.
- Big data techniques help financial firms improve the quality of the data they use in their analyses, which leads to more accurate results and enhanced business decisions.
Resources on big data in finance
- Mathematics, “Identifying Big Data’s Opportunities, Challenges, and Implications in Finance” — An examination of how big data has reshaped the finance industry and altered existing business models
- National Bureau of Economic Research, “Big Data in Finance” — A discussion of the impact of big data on corporate finance, market microstructure, asset pricing, and other areas of finance
Applications of big data analytics in finance
The many ways that firms are applying big data analytics in finance fall into three general categories:
- Techniques for improving the customer experience by using purchase histories, demographic data, and behavior tracking to offer personalized financial services, such as making product recommendations
- Methods for automating and otherwise enhancing the efficiency and quality of business processes, such as algorithmic trading process automation and credit risk determination
- Approaches to identifying and mitigating potential risks quickly to minimize financial exposure and capitalize on financial opportunities
Here’s a closer look at how three companies benefited from their application of big data analytics in their financial operations.
NASDAQ’s use of Amazon Web Services Simple Storage Service
NASDAQ is one of the largest securities markets in the world, handling share volumes that average around 4 billion exchanges each day. In 2014, the company switched from an on-premises data warehouse to Amazon Web Services (AWS). By 2020, NASDAQ’s AWS data warehouse processed financial data from thousands of sources totaling up to 70 billion records in a single day.
The use of AWS Simple Storage Service (S3) cloud storage technology allows NASDAQ to meet customer demand for fast access to historic stock information for its Market Replay and Data on Demand services. Market Replay allows clients to validate best execution and regulatory compliance by reconstructing events relating to a specific trade. Data on Demand provides NASDAQ traders with ready access to historical tick data for financial analytics purposes.
JP Morgan Chase’s use of Apache Hadoop
JP Morgan Chase is one of the largest firms in the financial industry, supporting approximately 3.5 billion user accounts and 30,000 separate databases. The company relies on the open-source Apache Hadoop big data framework to operate more efficiently by distributing processors and storage among low-cost hardware. The system collects and processes data from emails, social media posts, telephone calls, and other unstructured data sources.
Many of these data sources were unavailable to JP Morgan Chase prior to adopting the Hadoop framework, which limited its banking products’ effectiveness. Now the company’s data analytics operations more accurately reflect the attributes and tendencies of its millions of banking customers. As a result, its sales of foreclosed properties generate more revenue, and the bank is better able to assess credit to manage risk.
Acorns’ use of big data to revolutionize micro-investing
Acorns is one of the leading practitioners of automated micro-investing that combines automatic savings with portfolio management. The company uses machine learning techniques to identify customers’ spending patterns and automatically categorize their transactions. Clients are automatically notified when their spending increases, and the system can even recommend a budget.
The Acorns system works by collecting the excess “change” from customers’ credit card and online transactions and automatically depositing them in their investment portfolio. Clients have the option of applying the change automatically with each transaction or doing so manually on a per-transaction basis. Acorns’ robo-adviser applies algorithms to manage customers’ investment portfolios, which is much less expensive than relying on a human investment adviser.
Resources on big data analytics in finance
- SMB Compass, “Emerging Accounting Trends & Statistics for 2021” — An explanation of the impact of the COVID-19 pandemic on the adoption of big data and other technologies in accounting and finance
- Emerj, “Predictive Analytics in Finance — Current Applications and Trends” — A description of AI-based financial analysis applications from Teradata, Dataiku, DataRobot, and RapidMiner
Data mining in accounting
Data mining is the process of using software to identify patterns in large data repositories to learn more about a business’s customers, devise more effective marketing strategies, and operate more efficiently. Data mining in accounting extracts knowledge from huge stores of financial and other data to improve accounting practices’ effectiveness.
Three common applications of data mining in finance and accounting are to:
- Identify potential fraud
- Better organize accounting data
- Predict audit opinions on financial statements
Detecting fraud patterns in accounting databases
An important component of accounting data analytics is identifying potential fraud in financial records. Data mining tools spot outliers in massive pools of data that include atypical values and unusual behaviors. Among their applications are to detect symptoms of fraud in financial statements and to discover credit card fraud, securities fraud, corporate fraud, and other financial crimes.
Data mining in accounting has been shown to be more effective at detecting potential financial fraud than statistical methods because it applies machine learning to improve classification accuracy, especially when working with low sample data.
Enhanced categorization, clustering, and association of accounting data
Data mining tools use three different techniques to identify similarities within large data repositories:
- Categorization, also called classification, places each item contained in a dataset in a specific category, class, or group that shares characteristics and attributes.
- Clustering automatically creates a meaningful or useful cluster of data objects based on identifiable patterns.
- Association discovers patterns in the data based on relationships between the items in a single transaction, such as identifying products that consumers frequently purchase together.
Data mining techniques that predict audit opinions on financial statements
An accounting audit concludes with the auditor expressing one of two opinions:
- A qualified opinion indicates that the auditor found some material issues in the financial report in terms of the firm’s accounting policies but no misrepresentations of the company’s financial position.
- An unqualified opinion indicates that the company’s financial reports are fair and appropriate, in compliance, and without any exceptions.
Data mining tools can forecast the likelihood that an audit would result in one or the other opinion. Research published in the journal Mathematics showed that a model created using data mining techniques was able to predict the audit opinion of individual and consolidated financial statements with an accuracy of about 82.5%.
Resources on data mining in accounting
- Syntelli Solutions, “Using Big Data for Financial Fraud Prevention” — An examination of the relationship between data mining and financial fraud detection from the perspective of forensic accountants
- Financial Innovation, “Comprehensive Review of Text-Mining Applications in Finance” — Techniques for mining financial data in the form of unstructured text, such as press releases, news reports, and web pages
Accounting and data science
The union of accounting and data science has led to many of the principles of data analytics being applied to enhance accounting practices. Among the many ways that accountants apply data science techniques are to monitor and enhance accounting and financial processes, calculate the risk related to strategic decisions, and anticipate and meet their customers’ expectations.
Here’s a closer look at three examples of the use of data science to improve accounting and finance operations.
Helping an airline improve safety, reduce costs, and better serve customers
Competition in the airline industry is fierce, and airlines are among the most complex businesses to manage due to the many market variables and government regulations that can influence their profitability, as well as the industry’s high degree of unpredictability. Big data analytics and other data science concepts can increase airline revenue by providing companies with a greater understanding of customer behavior, more efficient maintenance schedules, and better fuel efficiency.
Sharing GPS Data with Banks to Prevent Fraud
Bank of America is one of several banks that are doing away with the traditional fraud alerts that notify customers when transactions occur far from the customer’s home. Instead, the bank uses the location services that accompany its mobile banking app, whose default settings include a daily location check, to verify that customers and their cards are in the same place. At present, the service is available only for the bank’s Visa card holders, but other banks are adopting the automated fraud detection technology as well.
Detecting fraud in credit card and banking payments
Data analytics, machine learning, and AI techniques are replacing the rules-based approach used previously by banks and credit card companies to detect payment fraud.
- Data credibility assessments are improved by automating gap analytics, which identifies missing values in sequences of transactions and automatically searches public data sources to fill in the gaps.
- Duplicate transactions are a common fraud method that charges the bank twice for a single transaction. AI-based systems are more accurate than rules-based approaches at discerning true accidental duplicate transactions from fraud attempts.
- Account theft and unusual transactions are typically thwarted by using behavior analysis: Consumers tend to follow similar patterns over time, so departures from those patterns are flagged as potential fraud. Machine learning algorithms are adept at analyzing behavior and deviations from that behavior to determine the likelihood of fraud.
Resources on accounting and data science
- DataToBiz, “Data Analytics Helping Accountants Excel! Role of Data Science in Accounting” — The outlook for data science in accounting from the perspective of a corporate chief financial officer
- Towards Data Science, “How AI Is Changing Financial Planning and Analysis” — A description of robotic process automation (RPA) as it applies to business processes, and financial analytics as an offshoot of predictive analytics and prescriptive analytics
How accounting data analytics benefits accountants
Data analytics presents accountants and finance professionals with an opportunity to regain some of the decision-making authority the professions had prior to the advent of automated decision-support systems over the past two decades. Accounting data has become one of several sources of information that contribute to a business’s analytics operations, and accountants have been relegated to providing only “historic” data while the analytics department provides insights and outlooks.
Greater insight into a company’s operations
Among the analytics skills that accountants and finance professionals need to develop to have a prominent role in a business’s strategic planning and forecasting are advanced revenue analytics, which focuses on pricing and sales channel optimization, and analytical segmentation, which helps companies align their marketing and sales strategies with their profit goals.
More accurate predictions via embedded predictive models
Companies are embedding predictive models in their business processes that can be expanded as new data sources become available. Embedded models can be updated more frequently based on the season, the accuracy of the existing models, and behavioral or other activity changes. As the models learn, they are better able to adapt automatically to unpredictable changes in markets, customer behavior, and other activities.
Increased automation of bookkeeping, compliance, and other accounting tasks
Automation continues to be applied to a growing number of business areas, including all aspects of accounting. For example, payroll automation is faster and more accurate than traditional payroll modules due to automated data input, net pay calculations, and data sharing. Similarly, by automating a business’s accounts receivable processes, accountants can include these records in their analytics operations more easily.
Growing importance of big data analytics in accounting and finance
The success of accountants and finance professionals depends increasingly on understanding the opportunities that data analytics creates for their clients and their industry. Accountants with a background in data analytics qualify for a far greater range of positions in accounting and finance. The growing adoption of data analytics in accounting and finance firms broadens the responsibilities of the professions while making their roles more important for supporting business decisions.