FINLLAMA: PARAMETER-EFFICIENT FINANCIAL SENTIMENT LEARNING USING LARGE LANGUAGE MODELS

The vast availability of textual data such as financial news, financial corporate announcements, earnings and macroeconomic news plays an important role in financial markets. The sentiment contained in unstructured textual information plays a critical role in market sentiment, returns and trading. As a result, financial sentiment analysis is an important area of study in the broader areas of Natural Language Processing (NLP) and financial analysis.Early approaches for financial sentiment analysis involved lexicon-based approaches that used lexicons of sentiment words to dictate sentiment. To overcome the shortcomings of the above approaches, machine learning techniques were introduced, which capable of acquiring sentiment from data-driven sources, but require data feature engineering and focus mainly on individual financial documents. The aim of this paper is to build and deploy a large-scale efficient financial sentiment analysis model using large language models based on the usage of the parameter efficient models of fine-tuning. The paper will involve the sentiment analysis of the articles for the financial news and classify them into the pre-defined categories of sentiments which are positive, neutral and negative. The paper involves acquiring and pre-processing the large scope financial news data namely by securing the Financial-News-Multisource data. The main tasks in the project include secure access to data, large-scale text processing, text cleaning, removing duplicates and dividing the data into chunks to be processed. It also includes automated sentiment labeling of data using a pre-trained model specific to the financial domain and low cost finetuning of a large language model using Low-Rank Adaptation (LoRA) approach to make the language model fit for the new domain. The project involves no domain related to forecasting financial market, no trading strategy and no deployment of trading system. They are also not manually labelled with sentiment information; sentiment labels are automatically generated instead to enable scalability.