via Cornell University
Machine learning can assess the effectiveness of mathematical tools used to predict the movements of financial markets, according to new Cornell research based on the largest dataset ever used in this area.
The researchers’ model could also predict future market movements, an extraordinarily difficult task because of markets’ massive amounts of information and high volatility.
“What we were trying to do is bring the power of machine learning techniques to not only evaluate how well our current methods and models work, but also to help us extend these in a way that we never could do without machine learning,” said Maureen O’Hara, the Robert W. Purcell Professor of Management at the SC Johnson College of Business.
O’Hara is co-author of “Microstructure in the Machine Age,” published July 7 in The Review of Financial Studies. Other Cornell co-authors are: David Easley, the Henry Scarborough Professor of Social Science in the College of Arts and Sciences and professor of information science in Computing and Information Science; and Marcos Lopez de Prado, professor of practice in Operations Research and Information Engineering in the College of Engineering and chief information officer of True Positive Technologies.
Zhibai Zhang of New York University’s Tandon School of Engineering, also a quantitative researcher at Bank of America, joined in the research.
“Trying to estimate these sorts of things using standard techniques gets very tricky, because the databases are so big. The beauty of machine learning is that it’s a different way to analyze the data,” O’Hara said. “The key thing we show in this paper is that in some cases, these microstructure features that attach to one contract are so powerful, they can predict the movements of other contracts. So we can pick up the patterns of how markets affect other markets, which is very difficult to do using standard tools.”
Markets generate vast amounts of data, and billions of dollars are at stake in mining that data for patterns to shed light on future market behavior. Companies on Wall Street and elsewhere employ various algorithms, examining different variables and factors, to find such patterns and predict the future.
In the study, the researchers used what’s known as a random forest machine learning algorithm to better understand the effectiveness of some of these models. They assessed the tools using a dataset of 87 futures contracts – agreements to buy or sell assets in the future at predetermined prices.
“Our sample is basically all active futures contracts around the world for five years, and we use every single trade – tens of millions of them – in our analysis,” O’Hara said. “What we did is use machine learning to try to understand how well microstructure tools developed for less complex market settings work to predict the future price process both within a contract and then collectively across contracts. We find that some of the variables work very, very well – and some of them not so great.”
Machine learning has long been used in finance, but typically as a so-called “black box” – in which an artificial intelligence algorithm uses reams of data to predict future patterns but without revealing how it makes its determinations. This method can be effective in the short term, O’Hara said, but sheds little light on what actually causes market patterns.
“Our use for machine learning is: I have a theory about what moves markets, so how can I test it?” she said. “How can I really understand whether my theories are any good? And how can I use what I learned from this machine learning approach to help me build better models and understand things that I can’t model because it’s too complex?”
Huge amounts of historical market data are available – every trade has been recorded since the 1980’s – and vast volumes of information are generated every day. Increased computing power and greater availability of data have made it possible to perform more fine-grained and comprehensive analyses, but these datasets, and the computing power needed to analyze them, can be prohibitively expensive for scholars.
In this research, finance industry practitioners partnered with the academic researchers to provide the data and the computers for the study as well as expertise in machine learning algorithms used in practice.
“This partnership brings benefits to both,” said O’Hara, adding that the paper is one in a line of research she, Easley and Lopez de Prado have completed over the last decade. “It allows us to do research in ways generally unavailable to academic researchers.”
The Latest Updates from Bing News & Google News
Go deeper with Bing News on:
Predicting market behavior
- Humanising Autonomy enhances behaviour AI for VisionTrackon May 18, 2021 at 5:14 am
Humanising Autonomy, a behaviour AI company, and VisionTrack, leading in-vehicle video telematics and computer vision provider, have entered into a strategic partnership to bring advanced vulnerable ...
- Q3, FY 2021 Results: Mytheresa Reports Exceptional Net Sales Growth of 47.5% in Q3, FY 2021 Benefiting From the Accelerated Shift in Consumer Behavioron May 18, 2021 at 3:00 am
Increase in net sales of 47.5% year-over-year to €164.8 million in Q3, FY 2021Acceleration evident by 66% two-year net sales growth compared to Q3, FY 2019Continued strong profitability with adjusted ...
- GMO’s Inker: 2 Predictions for How the Stock Bubble Could Endon May 14, 2021 at 11:56 am
Either way, the familiar assets stink and advisors should point clients elsewhere, Grantham's top asset allocator tells ThinkAdvisor.
- Market Research Tips From Industry Titans Aileen Lee, Brad Gerstner, And Joey Zwillingeron May 14, 2021 at 10:42 am
Investing in agile market research is one way to ensure your business stays ahead of the competition. All industries from B2B, B2C, financial services and beyond should prioritize market research not ...
- Setting The Record Straight On NFTs: Predictions About The Most Misunderstood Financial Advancement In Historyon May 13, 2021 at 4:45 pm
A “bubble” in an economic context, generally refers to a situation where the price for something—an individual stock, a financial asset, or even an entire market sector—exceeds its fundamental value ...
Go deeper with Google Headlines on:
Predicting market behavior
Go deeper with Bing News on:
Machine learning predictions
- One Drop brings AI-powered blood sugar predictions to Type 2 diabetes with European approvalon May 14, 2021 at 6:41 am
Using artificial intelligence to forecast and ward off oncoming changes in blood sugar levels has been a new frontier in Type 1 diabetes for years—but beyond that remains a largely untapped and ...
- The Quest for Objective Truth: Evolving from Opinions to Facts Using Machine Learning in Real Estateon May 13, 2021 at 9:34 am
In this contributed article, Anand Singh, a Senior Leader of Big Data Platform & Solutions at CoreLogic®, discusses how successfully drawing relevant conclusions from data requires human opinions and ...
- Deep Learning Analyzes CT Scans to Determine Cancer Typeon May 11, 2021 at 11:17 am
The deep learning network could help radiologists assess lung cancer types on CT scans, leading to more personalized treatment plans.
- Integration of machine learning and genome-scale metabolic modeling identifies multi-omics biomarkers for radiation resistanceon May 11, 2021 at 5:56 am
Personalized prediction of tumor radiosensitivity would facilitate development of precision medicine workflows for cancer treatment. Here, the authors integrate machine learning and genome-scale ...
- High Danger of Defect: Machine learning model predicts potential disk failures in Google’s DCson May 10, 2021 at 7:31 am
Google has devised a machine learning (ML) model that predicts disk failures with 98 per cent accuracy. The idea is to reduce data recovery work when disks actually fail. According to a Google blog by ...