Table of contents
AINewsTracker
AINewsTracker collects financial news, analyses article text and exposes the resulting data through FastAPI. A separate strategy path uses stored information for historical backtests.
1. Overview
app/main.py assembles the service, registers the route groups and configures Redis caching. News processing moves through a service, text metrics and the article repository before reaching MongoDB.
The companion article, Following financial news through AINewsTracker, follows that path from an incoming feed to a stored article.
2. Use cases
FastAPI service from AlgoÉTS that collects financial news, scores it with FinBERT and stores it in MongoDB.
- Fetch news from a source
- Quant analystNews feeds and APIs
The /news routes pull the Financial Modeling Prep 8-K RSS feed, GNews or a named feed with feedparser, download each article's text and save it; limits above 10 run as background tasks.
- Score article sentiment
- Quant analyst
TextMetrics loads ProsusAI/finbert for sentiment, Jean-Baptiste/roberta-ticker for ticker extraction and distilbart-cnn-12-6 for summaries; GET /articles/{article_id}/score returns the stored score.
- Query news by company
- Quant analyst
/company stores tickers, and /articles/articles_by_company_id/{company_id} or POST /articles/articles_by_company_ids return the articles linked to one or more companies.
- Seed companies and news
- Quant analystNews feeds and APIs
POST /seed/seed_companies and /seed/seed_news start CompanySeeder and NewsSeeder in the background; NewsSeeder reads Financial Modeling Prep stock news with FMP_API_KEY and saves the articles to MongoDB.
- Serve scored articles
- Backtesting service
GET /v1/articles, protected by a Bearer service token (AINEWSTRACKER_SERVICE_TOKEN), returns articles filtered by symbols and time window with sentiment label, probabilities and model name, paged by cursor.
2.1. Scenario: Collect and query Apple news locally
- Copy .env.example to .env, set FMP_API_KEY, then run docker-compose up --build to start the API with MongoDB and Redis.
- Open http://localhost:8000/api/v1/docs and call POST /seed/seed_companies.
- Call POST /seed/seed_news; NewsSeeder reads Financial Modeling Prep stock news in the background and stores the scored articles.
- Read the stored articles with GET /articles/ and check one with GET /articles/{article_id}/score.
- From a backtesting service, call GET /v1/articles?symbols=AAPL with the Bearer service token.
Files readREADME.mdapp/main.pyapp/routers/news.pyapp/routers/article.pyapp/routers/seed.pyapp/routers/integration.pyapp/core/services/news.pyapp/core/services/metrics.pydocker-compose.ymldocs/tickertools-integration.mdapp/seed/news.py.env.example
3. Getting started
cp .env.example .env
pip install poetry
poetry shell
poetry install
poetry run pytest
poetry run uvicorn app.main:app --reload
FastAPI documentation is available at http://localhost:8000/api/v1/docs according to the project guide. Docker users can start the documented stack with:
docker-compose up --build
4. Repository guide
app/main.pycreates the application, registers routers and configures caching.app/routers/defines news, article, company, user, seed and strategy endpoints.app/core/services/contains news processing and text metrics.app/core/repo/contains persistence operations grouped by domain.app/models/defines records accepted and returned by the service.app/seed/loads company and news data from external sources.app/core/trading/strategy.pyhandles strategy requests.
5. Features
- ingest news through RSS, GNews and configured feeds;
- analyse sentiment, topics and ticker allocation;
- manage article, company, news and user records;
- seed company and news data;
- execute strategy requests for one or more companies;
- run through Poetry or Docker Compose.
6. Architecture
This view condenses the runtime boundaries verified by GitDiagram. The full interactive diagram expands the model and links components to their source.
flowchart TD
Client(("API client")) --> App["FastAPI application<br/>app/main.py"]
App --> Routes["Routers<br/>app/routers/"]
Routes --> News["News processing<br/>app/core/services/news.py"]
News --> Metrics["Text metrics<br/>app/core/services/metrics.py"]
News --> Repos["Repositories<br/>app/core/repo/"]
Routes --> Repos
Repos --> Mongo[("MongoDB<br/>app/core/database/db.py")]
App --> Redis[("Redis cache<br/>app/core/database/db.py")]
Sources{{"Financial news sources"}} --> Seeders["Seeders<br/>app/seed/"]
Seeders --> News
Routes --> Strategy["Trading strategy<br/>app/core/trading/strategy.py"]
| Boundary | Responsibility | Evidence |
|---|---|---|
| Application | FastAPI lifecycle, router registration and cache setup | app/main.py |
| HTTP interface | News, article, company, user, seed and strategy routes | app/routers/ |
| Analysis | Feed processing and text metrics | app/core/services/news.py, app/core/services/metrics.py |
| Domain data | Article, company, news and user models | app/models/ |
| Persistence | MongoDB operations grouped by domain | app/core/repo/ |
| Operations | Data seeding and strategy execution | app/seed/, app/core/trading/strategy.py |
7. API contract
| Area | Documented operations |
|---|---|
| News | Create and read news; fetch RSS, GNews and feed entries |
| Articles | Create, list, read and delete articles; read an article score |
| Companies | Create companies, list companies and find one by ticker |
| Strategies | Execute a strategy for one company or a submitted company list |
The running FastAPI OpenAPI document is the authority for exact request and response schemas.
8. Deployment
The repository includes a root Dockerfile, a development docker-compose.yml and a separate deploy/ directory. Compose dependency ordering is a startup order, not a readiness guarantee.
9. Code structure
app/
├── main.py
├── core/{database,repo,services,telemetry,trading}/
├── models/
├── routers/
└── seed/
tests/
deploy/
monitoring/
10. Limits
- Sentiment and topic labels are analytical inputs, not financial advice or proof of causation.
- Historical backtests do not guarantee future performance.
- External feeds and market APIs have their own credentials, availability and usage terms.
- Endpoint details can change; current source and OpenAPI take precedence.
11. Sources
- GitHub repository
- Project documentation
- GitDiagram architecture
app/main.pyapp/routers/news.pyapp/core/services/news.pyapp/core/services/metrics.py
12. CI/CD pipeline
5 GitHub Actions workflows, 5 jobs. Each box is a job; double-bordered boxes publish or deploy, and arrows follow needs.
Mermaid Pipeline flow
flowchart LR
subgraph wf0["Python application CD"]
direction LR
wf0_setup[["setup<br/>ubuntu-latest"]]
end
subgraph wf1["Python application CI"]
direction LR
wf1_setup["setup<br/>ubuntu-latest"]
end
subgraph wf2["CodeQL"]
direction LR
wf2_analyze["Analyze<br/>ubuntu-latest"]
end
subgraph wf3["Docker CI"]
direction LR
wf3_docker[["docker<br/>ubuntu-latest"]]
end
subgraph wf4["AINewsTracker Docs"]
direction LR
wf4_build_and_deploy[["build_and_deploy<br/>ubuntu-latest"]]
end
t0(["release"])
t1(["pull request · push"])
t2(["push · pull request · schedule"])
t3(["push"])
t4(["push · pull request · manual run"])
d0[("Container registry")]
d1[("GitHub Pages")]
t0 --> wf0_setup
wf0_setup -.-> d0
t1 --> wf1_setup
t2 --> wf2_analyze
t3 --> wf3_docker
wf3_docker -.-> d0
t4 --> wf4_build_and_deploy
wf4_build_and_deploy -.-> d1
| Workflow | Triggers | Jobs | Publishes to |
|---|---|---|---|
Python application CD.github/workflows/cd.yml | release |
| Container registry |
Python application CI.github/workflows/ci.yml | pull request, push |
| — |
CodeQL.github/workflows/codeql.yml | push, pull request, schedule |
| — |
Docker CI.github/workflows/docker.yml | push |
| Container registry |
AINewsTracker Docs.github/workflows/docs.yml | push, pull request, manual run |
| GitHub Pages |
Generated from the source at commit 74a8366.
13. Deployment diagram
Mermaid Deployment diagram
flowchart LR
repo[("GitHub repository")]
ci[["GitHub Actions<br/>5 workflows"]]
image["Docker image<br/>FROM tiangolo/uvicorn-gunicorn-fastapi:python3.10"]
pub_Container_registry[("Container registry")]
pub_GitHub_Pages[("GitHub Pages")]
host_Vercel("Vercel<br/>vercel.json")
host_MkDocs_site("Documentation site (MkDocs)<br/>mkdocs.yml")
subgraph compose["Docker Compose"]
svc_redis["redis"]
svc_mongo["mongo"]
svc_web["web"]
svc_prometheus["prometheus"]
end
repo -->|"push or pull request"| ci
ci -->|"docker build"| image
image -->|"publishes"| pub_Container_registry
ci -->|"publishes"| pub_GitHub_Pages
ci -.->|"configured"| host_Vercel
ci -.->|"configured"| host_MkDocs_site
image -->|"docker compose up"| compose
14. External dependencies
127 declared packages (127 runtime, 0 development) in pyproject.toml, requirements.txt.
Package table 80 / 127
| Package | Version | Scope | Purpose | Registry |
|---|---|---|---|---|
fastapi | ^0.81.0 | runtime | Web framework | pypi.org |
pydantic | ^1.10.8 | runtime | Data validation | pypi.org |
python-multipart | ^0.0.6 | runtime | A streaming multipart parser for Python | pypi.org |
pymongo | ^4.3.3 | runtime | Database access | pypi.org |
python-dotenv | ^0.19.2 | runtime | Configuration | pypi.org |
uvicorn | — | runtime | Application server | pypi.org |
anyio | ^3.7.0 | runtime | High-level concurrency and networking framework on top of asyncio or Trio | pypi.org |
aioredis | ^2.0.0 | runtime | Database access | pypi.org |
configparser | ^5.3.0 | runtime | Configuration | pypi.org |
contractions | ^0.1.73 | runtime | Fixes contractions such as you're to you are | pypi.org |
fastapi_cache2 | ^0.2.1 | runtime | Cache for FastAPI | pypi.org |
fastapi_health | ^0.4.0 | runtime | Heath check on FastAPI applications. | pypi.org |
matplotlib | ^3.5.1 | runtime | Charts | pypi.org |
openpyxl | ^3.1.1 | runtime | Data analysis | pypi.org |
pandas | ^1.5.3 | runtime | Data analysis | pypi.org |
pandas-ta | ^0.3.14b | runtime | Data analysis | pypi.org |
prometheus-fastapi-instrumentator | ^5.9.1 | runtime | Logging and observability | pypi.org |
redis | ^4.5.1 | runtime | Database access | pypi.org |
rich | * | runtime | Command line | pypi.org |
structlog | ^23.1.0 | runtime | Logging and observability | pypi.org |
prometheus-client | ^0.17.0 | runtime | Logging and observability | pypi.org |
aiofiles | ^23.1.0 | runtime | File support for asyncio. | pypi.org |
types-aiofiles | ^23.1.0.4 | runtime | Type checking | pypi.org |
slowapi | ^0.1.8 | runtime | A rate limiting extension for Starlette and Fastapi | pypi.org |
asyncio | ^3.4.3 | runtime | Deprecated backport of asyncio; use the stdlib package instead | pypi.org |
jose | ^1.0.0 | runtime | Authentication and security | pypi.org |
pyjwt | ^2.7.0 | runtime | Authentication and security | pypi.org |
python-jose | ^3.3.0 | runtime | JOSE implementation in Python | pypi.org |
passlib | ^1.7.4 | runtime | comprehensive password hashing framework supporting over 30 schemes | pypi.org |
transformers | ^4.30.1 | runtime | Machine learning and AI | pypi.org |
torch | ^2.0.1 | runtime | Machine learning and AI | pypi.org |
motor | ^3.1.2 | runtime | Database access | pypi.org |
requests | ^2.31.0 | runtime | HTTP client | pypi.org |
beautifulsoup4 | ^4.12.2 | runtime | Scraping and parsing | pypi.org |
feedparser | ^6.0.10 | runtime | Scraping and parsing | pypi.org |
types-redis | ^4.5.5.2 | runtime | Type checking | pypi.org |
spacy | ^3.5.3 | runtime | Machine learning and AI | pypi.org |
finnhub-python | ^2.4.18 | runtime | Finnhub API | pypi.org |
aiohttp | ==3.8.4 | runtime | HTTP client | pypi.org |
aiosignal | ==1.3.1 | runtime | aiosignal: a list of registered asynchronous callbacks | pypi.org |
anyascii | ==0.3.2 | runtime | Unicode to ASCII transliteration | pypi.org |
async-timeout | ==4.0.2 | runtime | Timeout context manager for asyncio programs | pypi.org |
attrs | ==23.1.0 | runtime | Data validation | pypi.org |
blis | ==0.7.9 | runtime | The Blis BLAS-like linear algebra library, as a self-contained C-extension. | pypi.org |
catalogue | ==2.0.8 | runtime | Super lightweight function registries for your library | pypi.org |
certifi | ==2023.5.7 | runtime | Python package for providing Mozilla's CA Bundle. | pypi.org |
cffi | ==1.15.1 | runtime | Foreign Function Interface for Python calling C code. | pypi.org |
charset-normalizer | ==3.1.0 | runtime | The Real First Universal Charset Detector. Open, modern and actively maintained alternative to Chardet. | pypi.org |
click | ==8.1.3 | runtime | Command line | pypi.org |
colorama | ==0.4.6 | runtime | Command line | pypi.org |
confection | ==0.0.4 | runtime | The sweetest config system for Python | pypi.org |
contourpy | ==1.1.0 | runtime | Python library for calculating contours of 2D quadrilateral grids | pypi.org |
cryptography | ==41.0.1 | runtime | Authentication and security | pypi.org |
cycler | ==0.11.0 | runtime | Composable style cycles | pypi.org |
cymem | ==2.0.7 | runtime | Manage calls to calloc/free through Cython | pypi.org |
deprecated | ==1.2.14 | runtime | Python @deprecated decorator to deprecate old python classes, functions or methods. | pypi.org |
dnspython | ==2.3.0 | runtime | DNS toolkit | pypi.org |
ecdsa | ==0.18.0 | runtime | ECDSA cryptographic signature library (pure python) | pypi.org |
et-xmlfile | ==1.1.0 | runtime | An implementation of lxml.xmlfile for the standard library | pypi.org |
exceptiongroup | ==1.1.1 | runtime | Backport of PEP 654 (exception groups) | pypi.org |
fastapi-cache2 | ==0.2.1 | runtime | — | pypi.org |
fastapi-health | ==0.4.0 | runtime | — | pypi.org |
filelock | ==3.12.2 | runtime | A platform independent file lock. | pypi.org |
fonttools | ==4.40.0 | runtime | Tools to manipulate font files | pypi.org |
frozenlist | ==1.3.3 | runtime | A list-like structure which implements collections.abc.MutableSequence | pypi.org |
fsspec | ==2023.6.0 | runtime | File-system specification | pypi.org |
h11 | ==0.14.0 | runtime | A pure-Python, bring-your-own-I/O implementation of HTTP/1.1 | pypi.org |
huggingface-hub | ==0.15.1 | runtime | Machine learning and AI | pypi.org |
idna | ==3.4 | runtime | Internationalized Domain Names in Applications (IDNA) | pypi.org |
importlib-resources | ==5.12.0 | runtime | Read resources from Python packages | pypi.org |
jinja2 | ==3.1.2 | runtime | Templates | pypi.org |
kiwisolver | ==1.4.4 | runtime | A fast implementation of the Cassowary constraint solver | pypi.org |
langcodes | ==3.3.0 | runtime | Tools for labeling human languages with IETF language tags | pypi.org |
limits | ==3.5.0 | runtime | Rate limiting utilities | pypi.org |
markdown-it-py | ==3.0.0 | runtime | Python port of markdown-it. Markdown parsing, done right! | pypi.org |
markupsafe | ==2.1.3 | runtime | Safely add untrusted strings to HTML/XML markup. | pypi.org |
mdurl | ==0.1.2 | runtime | Markdown URL utilities | pypi.org |
mpmath | ==1.3.0 | runtime | Python library for arbitrary-precision floating-point arithmetic | pypi.org |
multidict | ==6.0.4 | runtime | — | pypi.org |
murmurhash | ==1.0.9 | runtime | Cython bindings for MurmurHash | pypi.org |
47 more packages are listed in the manifests.
15. Glossary
Repository
The versioned source files, configuration, and history of a project.
README
The repository’s entry document, usually covering its purpose and initial setup.
Dependency
A library, tool, or service that another component needs to build or run.
API
An interface through which software components exchange requests and results. An API does not have to use HTTP.
HTTP
A request-and-response protocol used by browsers and web services. A method, path, headers, and optional body describe a request.
Session
State kept across related requests, often to track a signed-in user. Authentication and permission checks remain separate concerns.
Container
An isolated process environment created from an image. Persistent data needs storage outside the container’s disposable filesystem.
Docker Compose
A configuration format and tool for starting related containers, networks, and volumes. Startup order alone does not guarantee service readiness.
Published port
A host port mapped to a service’s container port. Publishing a port can expose that service beyond its container network.
CLI
A text interface in which commands and arguments select operations. Exit codes let scripts detect success or failure.
Build
The steps that turn source files and dependencies into a runnable program or distributable artifact.
Mermaid
A text-based diagram language. Nodes and connections describe a model; the diagram itself does not execute the project.
Backend
An implementation behind a shared interface. Depending on context, it can mean a server-side service or a graphics, input, or storage implementation.
Database
A system for storing and querying structured records. Its schema, access rules, and persistence determine how the application can use that data.
16. References
- AlgoETS. (2026, November 5). AINewsTracker. [Software repository] github.com.
- Published documentation. (2026, November 5). [Web page] algoets.github.io.
- Interactive GitDiagram. (2026, November 5). [Web page] gitdiagram.com.
- AlgoETS. (2026, November 5). app/main.py. [Source file or documentation] github.com.
- AlgoETS. (2026, November 5). app/routers/news.py. [Source file or documentation] github.com.
- AlgoETS. (2026, November 5). app/core/services/news.py. [Source file or documentation] github.com.
- AlgoETS. (2026, November 5). app/core/services/metrics.py. [Source file or documentation] github.com.
Sources without a publication date show the date they were last checked, November 5, 2026. File links retain the inspected revision where available.