Building an AI-powered Financial Behavior Analyzer with NodeJS, Python, SvelteKit, and TailwindCSS - Part 1: The AI Service

Introduction

To kick off this intriguing and challenging journey, we'll start by building the AI service, the final component of our system’s architecture. We'll exploit the power of asynchronous programming in Python via the asyncio and one of its most lightweight yet powerful asynchronous HTTP Client/Servers, aiohttp. It is a "double-edged" sword that can be your http/websocket client and at the same time, http/websocket server. Unlike requests, which is synchronous, aiohttp supports both HTTP and WebSocket communication, making it a worthy alternative to Fast API, Flask, or even Django for async-based applications.

Let's put aiohttp to work in this article! 🚀

Prerequisite

Before diving in, I assume you've already:

Created a project (e.g., utility)

Set up a virtual environment

Installed the required dependencies

If you haven't, copy the following into requirements dot txt: The code below is 9 lines, from requirements dot txt.

and, while in your project's folder's virtual environment, run: The shell code below is 1 line.

Notice the versions of torch and torchvision used here. I used CPU-only versions of torch and torchvision since I deployed this on a CPU-based server. If you have a GPU-enabled server or development environment, you can install the standard versions without the URL specifier (at https: slash slash download dot pytorch dot org slash whl slash...) which, by the way, is a nifty way to include libraries' URL in requirements dot txt. Note: You may prefer to develop using the latest tourch (v2.6).

Just as I write this, The PyTorch Foundation announced the release of PyTorch® 2.6 which debuts it for Python 3.13 with some performance improvements. Consider using it instead.

Source code Sirneij/finance-analyzer

An AI-powered financial behavior analyzer and advisor written in Python (aiohttp) and TypeScript (ExpressJS & SvelteKit with Svelte 5) svelte typescript python javascript css3 html5

Implementation

Let's now go into the meat of this article.

Step 1: Spawning an aiohttp Server

Create app dot py and populate it with the following: The Python code below defines start background tasks, cleanup background tasks and cleanup ws, 140 lines, from app dot py.

It was a long line of code. However, looking closely, it's very straightforward. We started out registering all our WebSocket connections in a set (to prevent duplicates). We could have done something like as suggested: The Python code below is 7 lines, from app dot py.

but I went for the low-level approach and controlled the startup, and async locks to ensure thread safety, cleaning up, and stuff like that. You can use the suggested approach instead. Then comes the extract text route handler. It's the handler that receives the user's transactions PDF file, forces that file is its name (this is my preference, you can allow any file name), and then let extract text from pdf do its magic in extracting texts from the file and respond with the extracted text. Let's see how extract text from pdf can do this.

Step 2: Utility package - extract text dot py The Python code below defines extract text from pdf, 39 lines, from utils slash extract text dot py.

It uses Google OCR via the pytesseract library to extract texts from the file. Although, OCR should be able to get texts directly from any document but it worked better for me with images via image to string hence the intermediate step of converting the PDF file to images before feeding it into OCR. The temporary file creation was because convert from path requires a path that could only be gotten from a temporary file at this point since we don't want to save anyone's file on our end. A better option is to use the convert from bytes from pdf2image which doesn't require temporary file creation.

Back to app dot py, we also have a WebSocket handler that simply delegates actions to various submodules to handle. But before then, I have a utility that helps manage all the app's WebSocket connections. This is to elegantly manage connections without worrying about leaks and stuff like that.

Step 3: Utility package - websocket dot py The Python code below defines Web Socket Manager, init and prepare, 51 lines, from utils slash websocket dot py.

The app would well work without it but I encountered a bug where connections were not ready but requests were already being made by the frontend. So I figured having this utility would help to ensure the readiness of these connections before accepting requests. I also implemented methods, send result and send progress, to utilize aiohttp's WebSocket's send json to send analysis results and analysis progress reports respectively back to the requester(s) so that they won't be kept in the dark if the analysis is taking long (which will on a CPU-only machine using many transformer models). After the custom WebSocket manager completes preparation, we loop over the messages received and appropriately delegate actions based on the action key. For now, we support analyze and summary actions via the analyze transactions and summarize transactions respectively. Let's see what they do.

Step 4: Utility package - analyzer dot py The Python code below defines get device, analyze transactions and validate transaction, 299 lines, from utils slash analyzer dot py.

It's full of async functions that handle specific cases in the analysis (not minding transaction validation and device detection logic). A particular function of interest is the classify transactions which uses yiyanghkust slash finbert tone, A Large Language Model for Extracting Information from Financial Text, to perform Zero-Shot Classification of the transactions mostly into: The code below is 1 line, from dot env.

Realistically, it'd be more accurate to finetune a barebone Bidirectional Encoder Representations from Transformers (BERT) such as Ro BER Ta, Distil BERT, and co with our data but my excuse was lack of adequate data and GPU (Google Colab isn't enough). Tip: A nice way to collaborate

If you want to explore the idea behind finetuning and barebone Natural Language Processing with transformers, we can collaborate to create something like Fin BERT but for account statements and co. Please reach out.

There are other functions for detecting incomes/expenses that are "abnormal" by using IsolationForest and Z-score for the detection and for generating human-readable reasons for such a detection. Cool stuff!!! There were also functions for spending analysis and stuff but they are pretty basic. Note: amount assumption

All the analysis here assumed that income has a positive amount while expenses possess negative amounts. If your data is different from this assumption, you may need to modify the code or data depending on your preference.

Before we roundup, let's see what utils slash summarize dot py is

Step 5: Utility package - summarize dot py The Python code below defines summarize transactions, calculate trend and calculate percentage change, 200 lines, from utils slash summarize dot py.

Though long (due to repeated codes that could have been drafted into reusable functions), the function just does basic summaries of your transactions. It's comprehensive but can be well extended.

In both analyses, I endeavored to update connections with progress reports by sending progress actions at intervals informing the user of the time left.

Now, in app dot py, using init app, we created an Application instance and added routes to it. We could have used at route decorator instead. We didn't forget to do some housekeeping (clean-ups) of the background tasks previously started whenever we quit the app via different interrupts. That's it! Note: Undiscussed utilities

There were some submodules: utils slash settings dot py, models slash base dot py, etc, not discussed. They are basic setups for instrumentation (logging) and stuff. Others are for another system (maybe discussed in another series). In all, they are very basic.

Outro

Enjoyed this article? I'm a Software Engineer and Technical Writer actively seeking new opportunities, particularly in areas related to web security, finance, healthcare, and education. If you think my expertise aligns with your team's needs, let's chat! You can find me on LinkedIn and X. I am also an email away.

If you found this article valuable, consider sharing it with your network to help spread the knowledge!