Compare commits

..

6 Commits

Author SHA1 Message Date
Carlos Escalante
396c492f27 Record Phase 2 completion in PROGRESS.md
All checks were successful
Deploy to VPS / test (push) Successful in 1m40s
Deploy to VPS / deploy (push) Successful in 2m26s
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-10 15:31:42 -06:00
Carlos Escalante
440da3e394 Consistency sweep: enum members, keyword status codes, SimpleCookie
analytics.py filters by TransactionType.COMPRA instead of the string
literal; budget.py uses keyword HTTPException args; the agent
middleware parses cookies with stdlib SimpleCookie instead of a regex
(ARCH-10); auth.py's lazy imports are hoisted — there was never an
actual auth/db cycle (ARCH-20); transactions.py hoists or_/and_.
Verified: dev container boots, agent auth paths return 401 for
missing/garbage cookies.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-10 15:23:21 -06:00
Carlos Escalante
82f10a5d7c Error visibility and agent tool cleanup
Exchange-rate fetchers log every source failure instead of silently
passing (BE-18); the rate tool exposes fetched_at so staleness is
visible (BE-20). Agent: unbound-session failures raise a clear
RuntimeError (BE-12); net worth converts via get_crc_multipliers with
last-known fallbacks instead of a hardcoded 600 CRC / 1.08 EUR guess,
and reports accounts it cannot convert rather than inventing numbers
(BE-24); budget tool enforces MIN/MAX_YEAR (BE-16); municipal receipts
gain offset paging (BE-17); category analytics prefetches names; the
module docstring pins the read-only tool policy (SEC-09). Note: the
refresh loop never swallowed CancelledError (BaseException since 3.8) —
ARCH-08 was a false positive.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-10 15:23:21 -06:00
Carlos Escalante
82a7bfc4c5 Harden PDF uploads: size cap, magic-byte check, non-blocking parse
read_pdf_upload() bounds reads at 10 MB and requires the %PDF magic
before anything reaches pdftotext (SEC-06); rejections land in the
per-file errors list with the filename. Parsing now runs via
asyncio.to_thread so a slow PDF no longer blocks the event loop for up
to 30s (BE-14). Review notes: temp files were already context-managed
(BE-09 overstated) and the pension batch already commits atomically
(BE-10 overstated). 69 tests green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-10 15:00:49 -06:00
Carlos Escalante
3cdd7d9eab Constraint and query pass: FK delete rules, grouped queries, tiebreakers
- FK rules via migration 7884505b16b3 (verified on dev with a rolled-
  back probe): transactions/recurring items SET NULL on category delete
  (this also fixes the 500 that DELETE /categories raised for in-use
  categories), water readings CASCADE with their receipt. Models declare
  ondelete to stay in sync. (ARCH-09, BE-11)
- compute_actuals_by_source: 3 grouped queries replace 10 single-
  aggregate round-trips; behavior pinned by the existing cycle suite.
  (BE-21)
- compute_cc_by_category prefetches category names (ARCH-15); agent
  pension latest-per-fund resolved in SQL (BE-04); municipal water
  consumption batched into one grouped query (BE-05).
- Deterministic pagination: date DESC, id DESC on all transaction
  listings. (ARCH-12)
- New agent-tool tests (session binding, JSON-safe output, batched
  queries): 64 tests green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-10 14:45:46 -06:00
Carlos Escalante
74886fbf98 Migrate all 29 money columns from float to NUMERIC/Decimal (BE-02)
Models use Money = Annotated[Decimal, PlainSerializer(float)] so storage
is exact NUMERIC(15,2) (rates 15,6) while JSON keeps emitting plain
numbers — the frontend contract is unchanged, pinned by
test_models_serialization.py. Service layers coerce to float at their
boundaries (projection math, rate multipliers, agent tool output, the
savings constants become Decimal) so no naive Decimal/float mixing can
raise. Migration 7f567c8bafdb applied to dev: per-row values and the
table sum byte-identical before/after; endpoints verified live on the
new schema. 59/59 tests green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-10 13:38:28 -06:00
20 changed files with 903 additions and 243 deletions

View File

@@ -0,0 +1,61 @@
"""fk delete rules
Revision ID: 7884505b16b3
Revises: 7f567c8bafdb
Create Date: 2026-06-10 13:50:39.153585
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
import sqlmodel
# revision identifiers, used by Alembic.
revision: str = '7884505b16b3'
down_revision: Union[str, Sequence[str], None] = '7f567c8bafdb'
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
"""Add delete rules to the three FKs (review ARCH-09/BE-11).
Transactions and recurring items outlive their category (SET NULL);
water readings die with their receipt (CASCADE). Constraint names match
both pre-Alembic databases and fresh baseline builds (Postgres default
naming).
"""
op.drop_constraint("transaction_category_id_fkey", "transaction", type_="foreignkey")
op.create_foreign_key(
"transaction_category_id_fkey", "transaction", "category",
["category_id"], ["id"], ondelete="SET NULL",
)
op.drop_constraint("recurringitem_category_id_fkey", "recurringitem", type_="foreignkey")
op.create_foreign_key(
"recurringitem_category_id_fkey", "recurringitem", "category",
["category_id"], ["id"], ondelete="SET NULL",
)
op.drop_constraint("watermeterreading_receipt_id_fkey", "watermeterreading", type_="foreignkey")
op.create_foreign_key(
"watermeterreading_receipt_id_fkey", "watermeterreading", "municipalreceipt",
["receipt_id"], ["id"], ondelete="CASCADE",
)
def downgrade() -> None:
op.drop_constraint("watermeterreading_receipt_id_fkey", "watermeterreading", type_="foreignkey")
op.create_foreign_key(
"watermeterreading_receipt_id_fkey", "watermeterreading", "municipalreceipt",
["receipt_id"], ["id"],
)
op.drop_constraint("recurringitem_category_id_fkey", "recurringitem", type_="foreignkey")
op.create_foreign_key(
"recurringitem_category_id_fkey", "recurringitem", "category",
["category_id"], ["id"],
)
op.drop_constraint("transaction_category_id_fkey", "transaction", type_="foreignkey")
op.create_foreign_key(
"transaction_category_id_fkey", "transaction", "category",
["category_id"], ["id"],
)

View File

@@ -0,0 +1,263 @@
"""money columns to numeric
Revision ID: 7f567c8bafdb
Revises: c3da001a0eb3
Create Date: 2026-06-10 13:37:26.281634
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
import sqlmodel
# revision identifiers, used by Alembic.
revision: str = '7f567c8bafdb'
down_revision: Union[str, Sequence[str], None] = 'c3da001a0eb3'
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
"""Upgrade schema."""
# ### commands auto generated by Alembic - please adjust! ###
op.alter_column('account', 'balance',
existing_type=sa.DOUBLE_PRECISION(precision=53),
type_=sa.Numeric(precision=15, scale=2),
existing_nullable=False)
op.alter_column('account', 'next_payment',
existing_type=sa.DOUBLE_PRECISION(precision=53),
type_=sa.Numeric(precision=15, scale=2),
existing_nullable=True)
op.alter_column('balanceoverride', 'override_balance',
existing_type=sa.DOUBLE_PRECISION(precision=53),
type_=sa.Numeric(precision=15, scale=2),
existing_nullable=False)
op.alter_column('exchangerate', 'buy_rate',
existing_type=sa.DOUBLE_PRECISION(precision=53),
type_=sa.Numeric(precision=15, scale=6),
existing_nullable=False)
op.alter_column('exchangerate', 'sell_rate',
existing_type=sa.DOUBLE_PRECISION(precision=53),
type_=sa.Numeric(precision=15, scale=6),
existing_nullable=False)
op.alter_column('municipalreceipt', 'subtotal',
existing_type=sa.DOUBLE_PRECISION(precision=53),
type_=sa.Numeric(precision=15, scale=2),
existing_nullable=False)
op.alter_column('municipalreceipt', 'interests',
existing_type=sa.DOUBLE_PRECISION(precision=53),
type_=sa.Numeric(precision=15, scale=2),
existing_nullable=False)
op.alter_column('municipalreceipt', 'iva',
existing_type=sa.DOUBLE_PRECISION(precision=53),
type_=sa.Numeric(precision=15, scale=2),
existing_nullable=False)
op.alter_column('municipalreceipt', 'total',
existing_type=sa.DOUBLE_PRECISION(precision=53),
type_=sa.Numeric(precision=15, scale=2),
existing_nullable=False)
op.alter_column('pensionsnapshot', 'saldo_anterior',
existing_type=sa.DOUBLE_PRECISION(precision=53),
type_=sa.Numeric(precision=15, scale=2),
existing_nullable=False)
op.alter_column('pensionsnapshot', 'aportes',
existing_type=sa.DOUBLE_PRECISION(precision=53),
type_=sa.Numeric(precision=15, scale=2),
existing_nullable=False)
op.alter_column('pensionsnapshot', 'rendimientos',
existing_type=sa.DOUBLE_PRECISION(precision=53),
type_=sa.Numeric(precision=15, scale=2),
existing_nullable=False)
op.alter_column('pensionsnapshot', 'retiros',
existing_type=sa.DOUBLE_PRECISION(precision=53),
type_=sa.Numeric(precision=15, scale=2),
existing_nullable=False)
op.alter_column('pensionsnapshot', 'traslados',
existing_type=sa.DOUBLE_PRECISION(precision=53),
type_=sa.Numeric(precision=15, scale=2),
existing_nullable=False)
op.alter_column('pensionsnapshot', 'comision',
existing_type=sa.DOUBLE_PRECISION(precision=53),
type_=sa.Numeric(precision=15, scale=2),
existing_nullable=False)
op.alter_column('pensionsnapshot', 'correccion',
existing_type=sa.DOUBLE_PRECISION(precision=53),
type_=sa.Numeric(precision=15, scale=2),
existing_nullable=False)
op.alter_column('pensionsnapshot', 'bonificacion',
existing_type=sa.DOUBLE_PRECISION(precision=53),
type_=sa.Numeric(precision=15, scale=2),
existing_nullable=False)
op.alter_column('pensionsnapshot', 'saldo_final',
existing_type=sa.DOUBLE_PRECISION(precision=53),
type_=sa.Numeric(precision=15, scale=2),
existing_nullable=False)
op.alter_column('recurringitem', 'amount',
existing_type=sa.DOUBLE_PRECISION(precision=53),
type_=sa.Numeric(precision=15, scale=2),
existing_nullable=False)
op.alter_column('savingsaccrual', 'memp_amount',
existing_type=sa.DOUBLE_PRECISION(precision=53),
type_=sa.Numeric(precision=15, scale=2),
existing_nullable=False)
op.alter_column('savingsaccrual', 'mpat_amount',
existing_type=sa.DOUBLE_PRECISION(precision=53),
type_=sa.Numeric(precision=15, scale=2),
existing_nullable=False)
op.alter_column('transaction', 'amount',
existing_type=sa.DOUBLE_PRECISION(precision=53),
type_=sa.Numeric(precision=15, scale=2),
existing_nullable=False)
op.alter_column('watermeterreading', 'reading_previous',
existing_type=sa.DOUBLE_PRECISION(precision=53),
type_=sa.Numeric(precision=15, scale=2),
existing_nullable=False)
op.alter_column('watermeterreading', 'reading_current',
existing_type=sa.DOUBLE_PRECISION(precision=53),
type_=sa.Numeric(precision=15, scale=2),
existing_nullable=False)
op.alter_column('watermeterreading', 'consumption_m3',
existing_type=sa.DOUBLE_PRECISION(precision=53),
type_=sa.Numeric(precision=15, scale=2),
existing_nullable=False)
op.alter_column('watermeterreading', 'agua_potable',
existing_type=sa.DOUBLE_PRECISION(precision=53),
type_=sa.Numeric(precision=15, scale=2),
existing_nullable=False)
op.alter_column('watermeterreading', 'serv_ambientales',
existing_type=sa.DOUBLE_PRECISION(precision=53),
type_=sa.Numeric(precision=15, scale=2),
existing_nullable=False)
op.alter_column('watermeterreading', 'alcant_sanitario',
existing_type=sa.DOUBLE_PRECISION(precision=53),
type_=sa.Numeric(precision=15, scale=2),
existing_nullable=False)
op.alter_column('watermeterreading', 'iva',
existing_type=sa.DOUBLE_PRECISION(precision=53),
type_=sa.Numeric(precision=15, scale=2),
existing_nullable=False)
# ### end Alembic commands ###
def downgrade() -> None:
"""Downgrade schema."""
# ### commands auto generated by Alembic - please adjust! ###
op.alter_column('watermeterreading', 'iva',
existing_type=sa.Numeric(precision=15, scale=2),
type_=sa.DOUBLE_PRECISION(precision=53),
existing_nullable=False)
op.alter_column('watermeterreading', 'alcant_sanitario',
existing_type=sa.Numeric(precision=15, scale=2),
type_=sa.DOUBLE_PRECISION(precision=53),
existing_nullable=False)
op.alter_column('watermeterreading', 'serv_ambientales',
existing_type=sa.Numeric(precision=15, scale=2),
type_=sa.DOUBLE_PRECISION(precision=53),
existing_nullable=False)
op.alter_column('watermeterreading', 'agua_potable',
existing_type=sa.Numeric(precision=15, scale=2),
type_=sa.DOUBLE_PRECISION(precision=53),
existing_nullable=False)
op.alter_column('watermeterreading', 'consumption_m3',
existing_type=sa.Numeric(precision=15, scale=2),
type_=sa.DOUBLE_PRECISION(precision=53),
existing_nullable=False)
op.alter_column('watermeterreading', 'reading_current',
existing_type=sa.Numeric(precision=15, scale=2),
type_=sa.DOUBLE_PRECISION(precision=53),
existing_nullable=False)
op.alter_column('watermeterreading', 'reading_previous',
existing_type=sa.Numeric(precision=15, scale=2),
type_=sa.DOUBLE_PRECISION(precision=53),
existing_nullable=False)
op.alter_column('transaction', 'amount',
existing_type=sa.Numeric(precision=15, scale=2),
type_=sa.DOUBLE_PRECISION(precision=53),
existing_nullable=False)
op.alter_column('savingsaccrual', 'mpat_amount',
existing_type=sa.Numeric(precision=15, scale=2),
type_=sa.DOUBLE_PRECISION(precision=53),
existing_nullable=False)
op.alter_column('savingsaccrual', 'memp_amount',
existing_type=sa.Numeric(precision=15, scale=2),
type_=sa.DOUBLE_PRECISION(precision=53),
existing_nullable=False)
op.alter_column('recurringitem', 'amount',
existing_type=sa.Numeric(precision=15, scale=2),
type_=sa.DOUBLE_PRECISION(precision=53),
existing_nullable=False)
op.alter_column('pensionsnapshot', 'saldo_final',
existing_type=sa.Numeric(precision=15, scale=2),
type_=sa.DOUBLE_PRECISION(precision=53),
existing_nullable=False)
op.alter_column('pensionsnapshot', 'bonificacion',
existing_type=sa.Numeric(precision=15, scale=2),
type_=sa.DOUBLE_PRECISION(precision=53),
existing_nullable=False)
op.alter_column('pensionsnapshot', 'correccion',
existing_type=sa.Numeric(precision=15, scale=2),
type_=sa.DOUBLE_PRECISION(precision=53),
existing_nullable=False)
op.alter_column('pensionsnapshot', 'comision',
existing_type=sa.Numeric(precision=15, scale=2),
type_=sa.DOUBLE_PRECISION(precision=53),
existing_nullable=False)
op.alter_column('pensionsnapshot', 'traslados',
existing_type=sa.Numeric(precision=15, scale=2),
type_=sa.DOUBLE_PRECISION(precision=53),
existing_nullable=False)
op.alter_column('pensionsnapshot', 'retiros',
existing_type=sa.Numeric(precision=15, scale=2),
type_=sa.DOUBLE_PRECISION(precision=53),
existing_nullable=False)
op.alter_column('pensionsnapshot', 'rendimientos',
existing_type=sa.Numeric(precision=15, scale=2),
type_=sa.DOUBLE_PRECISION(precision=53),
existing_nullable=False)
op.alter_column('pensionsnapshot', 'aportes',
existing_type=sa.Numeric(precision=15, scale=2),
type_=sa.DOUBLE_PRECISION(precision=53),
existing_nullable=False)
op.alter_column('pensionsnapshot', 'saldo_anterior',
existing_type=sa.Numeric(precision=15, scale=2),
type_=sa.DOUBLE_PRECISION(precision=53),
existing_nullable=False)
op.alter_column('municipalreceipt', 'total',
existing_type=sa.Numeric(precision=15, scale=2),
type_=sa.DOUBLE_PRECISION(precision=53),
existing_nullable=False)
op.alter_column('municipalreceipt', 'iva',
existing_type=sa.Numeric(precision=15, scale=2),
type_=sa.DOUBLE_PRECISION(precision=53),
existing_nullable=False)
op.alter_column('municipalreceipt', 'interests',
existing_type=sa.Numeric(precision=15, scale=2),
type_=sa.DOUBLE_PRECISION(precision=53),
existing_nullable=False)
op.alter_column('municipalreceipt', 'subtotal',
existing_type=sa.Numeric(precision=15, scale=2),
type_=sa.DOUBLE_PRECISION(precision=53),
existing_nullable=False)
op.alter_column('exchangerate', 'sell_rate',
existing_type=sa.Numeric(precision=15, scale=6),
type_=sa.DOUBLE_PRECISION(precision=53),
existing_nullable=False)
op.alter_column('exchangerate', 'buy_rate',
existing_type=sa.Numeric(precision=15, scale=6),
type_=sa.DOUBLE_PRECISION(precision=53),
existing_nullable=False)
op.alter_column('balanceoverride', 'override_balance',
existing_type=sa.Numeric(precision=15, scale=2),
type_=sa.DOUBLE_PRECISION(precision=53),
existing_nullable=False)
op.alter_column('account', 'next_payment',
existing_type=sa.Numeric(precision=15, scale=2),
type_=sa.DOUBLE_PRECISION(precision=53),
existing_nullable=True)
op.alter_column('account', 'balance',
existing_type=sa.Numeric(precision=15, scale=2),
type_=sa.DOUBLE_PRECISION(precision=53),
existing_nullable=False)
# ### end Alembic commands ###

View File

@@ -3,6 +3,11 @@ Read-only tools exposed to the MAF ChatAgent. Each tool is a thin wrapper
around existing SQLModel queries / service helpers — they do NOT duplicate
business logic. The active DB session is resolved via a ContextVar so tool
signatures stay clean for the LLM.
POLICY: every tool here must stay READ-ONLY. Transaction descriptions reach
the model from external emails (indirect prompt-injection surface), so a
write-capable tool would let crafted text mutate financial data. Any future
write tool requires an explicit user-confirmation step in the UI.
"""
from __future__ import annotations
@@ -28,10 +33,13 @@ from app.models.models import (
WaterMeterReading,
)
from app.services.budget_projection import (
MAX_YEAR,
MIN_YEAR,
compute_monthly_projection,
compute_yearly_projection_with_cumulative,
get_cycle_range,
)
from app.services import exchange_rate as fx
from app.services.exchange_rate import (
get_converted_amount_expr,
get_current_rate,
@@ -49,7 +57,13 @@ def reset_session(token: contextvars.Token) -> None:
def _s() -> Session:
return _session_ctx.get()
try:
return _session_ctx.get()
except LookupError as exc:
raise RuntimeError(
"DB session not bound to agent context — tool called outside an "
"HTTP request (the AG-UI middleware binds it per request)"
) from exc
# ─── Tools ──────────────────────────────────────────────────────────────────
@@ -66,9 +80,9 @@ def get_accounts() -> list[dict]:
"bank": a.bank.value,
"label": a.label,
"currency": a.currency.value,
"balance": a.balance,
"balance": float(a.balance),
"account_type": a.account_type.value,
"next_payment": a.next_payment,
"next_payment": float(a.next_payment) if a.next_payment is not None else None,
}
for a in rows
]
@@ -77,26 +91,31 @@ def get_accounts() -> list[dict]:
def get_net_worth() -> dict:
"""Return total assets, liabilities and net worth in CRC (primary currency).
USD/EUR balances are converted at the latest exchange rate."""
accounts = _s().exec(select(Account)).all()
rate = get_current_rate(_s())
sell = rate.sell_rate if rate else 600.0
session = _s()
accounts = session.exec(select(Account)).all()
multipliers = fx.get_crc_multipliers(session)
assets_crc = 0.0
liabilities_crc = 0.0
excluded: list[str] = []
for a in accounts:
amt = a.balance
if a.currency.value == "USD":
amt = a.balance * sell
elif a.currency.value == "EUR":
amt = a.balance * sell * 1.08 # rough; real conversion is endpoint-side
mult = multipliers.get(a.currency.value)
if mult is None:
# No live or last-known rate: say so instead of inventing a number
excluded.append(f"{a.label} ({a.currency.value})")
continue
amt = float(a.balance) * float(mult)
if a.account_type.value == "LIABILITY":
liabilities_crc += amt
else:
assets_crc += amt
return {
result = {
"assets_crc": round(assets_crc, 2),
"liabilities_crc": round(liabilities_crc, 2),
"net_crc": round(assets_crc - liabilities_crc, 2),
}
if excluded:
result["excluded_accounts_no_rate"] = excluded
return result
def get_recent_transactions(
@@ -139,7 +158,7 @@ def get_recent_transactions(
"id": t.id,
"date": t.date.isoformat(),
"merchant": t.merchant,
"amount": t.amount,
"amount": float(t.amount),
"currency": t.currency.value,
"source": t.source.value,
"transaction_type": t.transaction_type.value,
@@ -218,6 +237,8 @@ def get_budget_projection(
"""Budget projection. If month is omitted, returns the yearly rollup; if
given, returns the monthly detail with income items, expense items and
actuals by source."""
if not MIN_YEAR <= year <= MAX_YEAR:
return {"error": f"year must be between {MIN_YEAR} and {MAX_YEAR}"}
session = _s()
if month is None:
months_data = compute_yearly_projection_with_cumulative(session, year)
@@ -243,7 +264,7 @@ def list_recurring_items() -> list[dict]:
{
"id": r.id,
"name": r.name,
"amount": r.amount,
"amount": float(r.amount),
"currency": r.currency.value,
"item_type": r.item_type.value,
"frequency": r.frequency.value,
@@ -266,27 +287,38 @@ def get_pension_snapshots(
) -> list[dict]:
"""Pension fund snapshots. Each snapshot covers a period with balances,
contributions, returns, fees and the ending balance (saldo_final)."""
q = select(PensionSnapshot).order_by(col(PensionSnapshot.period_end).desc())
if latest_only:
# Latest snapshot per fund resolved in SQL instead of scanning all rows
latest = (
select(
PensionSnapshot.fund.label("fund"),
func.max(PensionSnapshot.period_end).label("period_end"),
)
.group_by(PensionSnapshot.fund)
.subquery()
)
q = select(PensionSnapshot).join(
latest,
(PensionSnapshot.fund == latest.c.fund)
& (PensionSnapshot.period_end == latest.c.period_end),
)
else:
q = select(PensionSnapshot)
q = q.order_by(col(PensionSnapshot.period_end).desc())
if fund:
q = q.where(PensionSnapshot.fund == fund)
rows = _s().exec(q).all()
if latest_only:
seen: dict[str, PensionSnapshot] = {}
for r in rows:
if r.fund.value not in seen:
seen[r.fund.value] = r
rows = list(seen.values())
return [
{
"fund": r.fund.value,
"period_start": r.period_start.isoformat(),
"period_end": r.period_end.isoformat(),
"saldo_anterior": r.saldo_anterior,
"aportes": r.aportes,
"rendimientos": r.rendimientos,
"retiros": r.retiros,
"comision": r.comision,
"saldo_final": r.saldo_final,
"saldo_anterior": float(r.saldo_anterior),
"aportes": float(r.aportes),
"rendimientos": float(r.rendimientos),
"retiros": float(r.retiros),
"comision": float(r.comision),
"saldo_final": float(r.saldo_final),
}
for r in rows
]
@@ -311,6 +343,7 @@ def get_salary_summary() -> dict:
def get_municipal_receipts(
limit: Annotated[int, Field(ge=1, le=50)] = 12,
offset: Annotated[int, Field(ge=0, description="Skip the N most recent")] = 0,
account: Annotated[
Optional[str], Field(description="Municipal account/contract id")
] = None,
@@ -320,13 +353,23 @@ def get_municipal_receipts(
q = select(MunicipalReceipt).order_by(col(MunicipalReceipt.receipt_date).desc())
if account:
q = q.where(MunicipalReceipt.account == account)
q = q.limit(limit)
q = q.offset(offset).limit(limit)
rows = _s().exec(q).all()
# One grouped query for all receipts instead of one per receipt
consumption: dict[int, float] = {}
ids = [r.id for r in rows if r.id is not None]
if ids:
grouped = _s().exec(
select(
WaterMeterReading.receipt_id,
func.sum(WaterMeterReading.consumption_m3),
)
.where(col(WaterMeterReading.receipt_id).in_(ids))
.group_by(WaterMeterReading.receipt_id)
).all()
consumption = {rid: float(total) for rid, total in grouped}
out: list[dict] = []
for r in rows:
readings = _s().exec(
select(WaterMeterReading).where(WaterMeterReading.receipt_id == r.id)
).all()
out.append(
{
"id": r.id,
@@ -334,11 +377,11 @@ def get_municipal_receipts(
"period": r.period,
"account": r.account,
"finca": r.finca,
"subtotal": r.subtotal,
"interests": r.interests,
"iva": r.iva,
"total": r.total,
"water_consumption_m3": sum(w.consumption_m3 for w in readings),
"subtotal": float(r.subtotal),
"interests": float(r.interests),
"iva": float(r.iva),
"total": float(r.total),
"water_consumption_m3": consumption.get(r.id, 0.0),
}
)
return out
@@ -366,13 +409,10 @@ def get_analytics_by_category(
q = q.where(Transaction.date >= start, Transaction.date < end)
rows = session.exec(q).all()
grand = sum(float(r[1]) for r in rows) or 1.0
names = {c.id: c.name for c in session.exec(select(Category)).all()}
out = []
for cat_id, total, count in rows:
name = "Uncategorized"
if cat_id:
cat = session.get(Category, cat_id)
if cat:
name = cat.name
name = names.get(cat_id, "Uncategorized") if cat_id else "Uncategorized"
out.append(
{
"category_id": cat_id,
@@ -436,9 +476,10 @@ def get_exchange_rate() -> dict:
if not rate:
return {"buy_rate": None, "sell_rate": None, "date": None}
return {
"buy_rate": rate.buy_rate,
"sell_rate": rate.sell_rate,
"buy_rate": float(rate.buy_rate),
"sell_rate": float(rate.sell_rate),
"date": rate.date.isoformat(),
"fetched_at": rate.fetched_at.isoformat() if rate.fetched_at else None,
}

View File

@@ -8,7 +8,7 @@ from sqlmodel import Session, func, select
from app.auth import get_current_user
from app.db import get_session
from app.models.models import Category, Transaction
from app.models.models import Category, Transaction, TransactionType
from app.services.budget_projection import get_cycle_range
from app.services.exchange_rate import get_converted_amount_expr
@@ -53,7 +53,7 @@ def spending_by_category(
func.sum(amount_crc).label("total"),
func.count().label("count"),
)
.where(Transaction.transaction_type == "COMPRA")
.where(Transaction.transaction_type == TransactionType.COMPRA)
.group_by(Transaction.category_id)
)
@@ -123,7 +123,7 @@ def monthly_trend(
),
)
.where(
Transaction.transaction_type == "COMPRA",
Transaction.transaction_type == TransactionType.COMPRA,
Transaction.date >= start,
Transaction.date < end,
)
@@ -171,7 +171,7 @@ def daily_spending(
func.sum(amount_crc).label("total"),
func.count().label("count"),
)
.where(Transaction.transaction_type == "COMPRA")
.where(Transaction.transaction_type == TransactionType.COMPRA)
.group_by(func.date(Transaction.date))
.order_by(func.date(Transaction.date))
)

View File

@@ -248,9 +248,9 @@ def upsert_balance_override(
_user: str = Depends(get_current_user),
):
if year < MIN_YEAR or year > MAX_YEAR:
raise HTTPException(400, f"Year must be between {MIN_YEAR} and {MAX_YEAR}")
raise HTTPException(status_code=400, detail=f"Year must be between {MIN_YEAR} and {MAX_YEAR}")
if year == FRESH_START_YEAR and month < FRESH_START_MONTH:
raise HTTPException(400, f"Cannot override before {FRESH_START_YEAR}-{FRESH_START_MONTH:02d}")
raise HTTPException(status_code=400, detail=f"Cannot override before {FRESH_START_YEAR}-{FRESH_START_MONTH:02d}")
existing = session.exec(
select(BalanceOverride).where(
@@ -288,6 +288,6 @@ def delete_balance_override(
)
).first()
if not existing:
raise HTTPException(404, "No override found for this month")
raise HTTPException(status_code=404, detail="No override found for this month")
session.delete(existing)
session.commit()

View File

@@ -2,11 +2,14 @@ from datetime import datetime
from dateutil.relativedelta import relativedelta
from typing import Optional
import asyncio
from fastapi import APIRouter, Depends, Query, UploadFile
from pydantic import BaseModel
from sqlmodel import Session, select
from app.auth import get_current_user
from app.uploads import read_pdf_upload
from app.db import get_session
from app.models.models import (
Category,
@@ -204,8 +207,9 @@ async def upload_municipal_receipt(
errors: list[str] = []
try:
pdf_bytes = await file.read()
data = extract_municipal_receipt(pdf_bytes, filename)
pdf_bytes = await read_pdf_upload(file)
# pdftotext runs as a subprocess; keep the event loop free
data = await asyncio.to_thread(extract_municipal_receipt, pdf_bytes, filename)
except ValueError as e:
return MunicipalReceiptUploadResult(imported=0, updated=0, errors=[str(e)])
except Exception as e:

View File

@@ -1,10 +1,13 @@
from datetime import date
import asyncio
from fastapi import APIRouter, Depends, UploadFile
from pydantic import BaseModel
from sqlmodel import Session, select
from app.auth import get_current_user
from app.uploads import read_pdf_upload
from app.db import get_session
from app.models.models import Bank, PensionSnapshot, PensionSnapshotRead
from app.services.pension_pdf import parse_pension_pdf
@@ -115,8 +118,11 @@ async def upload_pension_pdfs(
for file in files:
filename = file.filename or "unknown.pdf"
try:
pdf_bytes = await file.read()
fund_snapshots = parse_pension_pdf(pdf_bytes, filename)
pdf_bytes = await read_pdf_upload(file)
# pdftotext runs as a subprocess (up to 30s); keep the event loop free
fund_snapshots = await asyncio.to_thread(
parse_pension_pdf, pdf_bytes, filename
)
except ValueError as e:
errors.append(str(e))
continue

View File

@@ -31,7 +31,7 @@ def list_salarios(
query = (
select(Transaction)
.where(col(Transaction.transaction_type).in_(SALARIO_TYPES))
.order_by(col(Transaction.date).desc())
.order_by(col(Transaction.date).desc(), col(Transaction.id).desc())
.offset(offset)
.limit(limit)
)

View File

@@ -3,6 +3,7 @@ from typing import Optional
from fastapi import APIRouter, Depends, HTTPException, Query
from pydantic import BaseModel
from sqlalchemy import and_, or_
from sqlmodel import Session, col, func, select
from app.auth import get_current_user
@@ -73,7 +74,6 @@ def list_transactions(
prev_y, prev_m = get_previous_cycle(cycle_year, cycle_month)
prev_start, prev_end = get_cycle_range(prev_y, prev_m)
# Normal transactions in this cycle (not deferred) + deferred from previous cycle
from sqlalchemy import or_, and_
query = query.where(
or_(
and_(
@@ -93,7 +93,7 @@ def list_transactions(
Transaction.date >= datetime.fromisoformat(start_date),
Transaction.date < datetime.fromisoformat(end_date),
)
query = query.order_by(col(Transaction.date).desc()).offset(offset).limit(limit)
query = query.order_by(col(Transaction.date).desc(), col(Transaction.id).desc()).offset(offset).limit(limit)
return session.exec(query).all()
@@ -166,7 +166,7 @@ def recent_transactions(
query = (
select(Transaction)
.where(Transaction.source == TransactionSource.CREDIT_CARD)
.order_by(col(Transaction.date).desc())
.order_by(col(Transaction.date).desc(), col(Transaction.id).desc())
.limit(limit)
)
return session.exec(query).all()

View File

@@ -12,6 +12,8 @@ from jose import JWTError, jwt
from sqlmodel import Session, select
from app.config import settings
from app.db import get_session
from app.models.models import APIToken
from app.timeutil import utcnow
oauth2_scheme = OAuth2PasswordBearer(tokenUrl="/api/v1/auth/login")
@@ -91,9 +93,6 @@ def _validate_token(token: str) -> str:
pass
# Fallback: check API token
from app.db import get_session
from app.models.models import APIToken
token_hash = hash_token(token)
with next(get_session()) as session:
api_token = session.exec(

View File

@@ -1,7 +1,7 @@
import asyncio
import json
import re
import uuid
from http.cookies import SimpleCookie
from contextlib import asynccontextmanager
from agent_framework_ag_ui import add_agent_framework_fastapi_endpoint
@@ -105,10 +105,10 @@ async def agent_auth_and_session(request: Request, call_next):
if auth_header.lower().startswith("bearer "):
token = auth_header.split(" ", 1)[1].strip()
else:
cookie_header = request.headers.get("cookie", "")
m = re.search(r"(?:^|;\s*)ws_token=([^;]+)", cookie_header)
if m:
token = m.group(1)
cookies = SimpleCookie()
cookies.load(request.headers.get("cookie", ""))
if "ws_token" in cookies:
token = cookies["ws_token"].value
if not token:
return Response(status_code=401, content="Missing auth")

View File

@@ -1,7 +1,9 @@
import enum
from datetime import date, datetime
from typing import Optional
from decimal import Decimal
from typing import Annotated, Optional
from pydantic import PlainSerializer
from sqlalchemy import JSON, Column, UniqueConstraint
from sqlalchemy.dialects.postgresql import JSONB
from sqlmodel import Field, Relationship, SQLModel
@@ -12,6 +14,13 @@ from app.timeutil import utcnow
# SQLite (tests).
JSON_OR_JSONB = JSON().with_variant(JSONB(), "postgresql")
# Monetary values: exact NUMERIC storage, but serialized to JSON as plain
# numbers so the API contract (frontend expects `number`) is unchanged.
# Pinned by tests/test_models_serialization.py.
Money = Annotated[
Decimal, PlainSerializer(float, return_type=float, when_used="json")
]
class RecurringItemType(str, enum.Enum):
INCOME = "INCOME"
@@ -106,9 +115,9 @@ class AccountBase(SQLModel):
bank: Bank
currency: Currency
label: str
balance: float = 0.0
balance: Money = Field(default=Decimal("0"), max_digits=15, decimal_places=2)
account_type: AccountType = AccountType.BANK
next_payment: Optional[float] = None
next_payment: Optional[Money] = Field(default=None, max_digits=15, decimal_places=2)
class Account(AccountBase, table=True):
@@ -126,16 +135,16 @@ class AccountRead(AccountBase):
class AccountUpdate(SQLModel):
balance: Optional[float] = None
balance: Optional[Money] = None
label: Optional[str] = None
next_payment: Optional[float] = None
next_payment: Optional[Money] = None
# --- Transaction ---
class TransactionBase(SQLModel):
amount: float
amount: Money = Field(max_digits=15, decimal_places=2)
currency: Currency = Currency.CRC
merchant: str
city: Optional[str] = None
@@ -148,7 +157,9 @@ class TransactionBase(SQLModel):
source: TransactionSource = TransactionSource.CREDIT_CARD
bank: Bank = Bank.BAC
notes: Optional[str] = None
category_id: Optional[int] = Field(default=None, foreign_key="category.id")
category_id: Optional[int] = Field(
default=None, foreign_key="category.id", ondelete="SET NULL"
)
deferred_to_next_cycle: bool = Field(
default=False, sa_column_kwargs={"server_default": "false"}
)
@@ -171,7 +182,7 @@ class TransactionRead(TransactionBase):
class TransactionUpdate(SQLModel):
amount: Optional[float] = None
amount: Optional[Money] = None
currency: Optional[Currency] = None
merchant: Optional[str] = None
city: Optional[str] = None
@@ -189,14 +200,14 @@ class TransactionUpdate(SQLModel):
class ExchangeRate(SQLModel, table=True):
id: Optional[int] = Field(default=None, primary_key=True)
date: datetime
buy_rate: float
sell_rate: float
buy_rate: Decimal = Field(max_digits=15, decimal_places=6)
sell_rate: Decimal = Field(max_digits=15, decimal_places=6)
fetched_at: datetime = Field(default_factory=utcnow)
class ExchangeRateRead(SQLModel):
buy_rate: float
sell_rate: float
buy_rate: Money
sell_rate: Money
date: datetime
fetched_at: datetime
@@ -254,7 +265,7 @@ class UserSettingsUpdate(SQLModel):
class RecurringItemBase(SQLModel):
name: str
amount: float
amount: Money = Field(max_digits=15, decimal_places=2)
currency: Currency = Currency.CRC
item_type: RecurringItemType
frequency: RecurringFrequency = RecurringFrequency.MONTHLY
@@ -264,7 +275,9 @@ class RecurringItemBase(SQLModel):
default=None,
sa_column=Column(JSON, nullable=True),
)
category_id: Optional[int] = Field(default=None, foreign_key="category.id")
category_id: Optional[int] = Field(
default=None, foreign_key="category.id", ondelete="SET NULL"
)
is_active: bool = True
notes: Optional[str] = None
@@ -287,7 +300,7 @@ class RecurringItemRead(RecurringItemBase):
class RecurringItemUpdate(SQLModel):
name: Optional[str] = None
amount: Optional[float] = None
amount: Optional[Money] = None
currency: Optional[Currency] = None
item_type: Optional[RecurringItemType] = None
frequency: Optional[RecurringFrequency] = None
@@ -323,15 +336,15 @@ class PensionSnapshotBase(SQLModel):
contract_number: str
period_start: date
period_end: date
saldo_anterior: float
aportes: float
rendimientos: float
retiros: float
traslados: float
comision: float
correccion: float
bonificacion: float
saldo_final: float
saldo_anterior: Money = Field(max_digits=15, decimal_places=2)
aportes: Money = Field(max_digits=15, decimal_places=2)
rendimientos: Money = Field(max_digits=15, decimal_places=2)
retiros: Money = Field(max_digits=15, decimal_places=2)
traslados: Money = Field(max_digits=15, decimal_places=2)
comision: Money = Field(max_digits=15, decimal_places=2)
correccion: Money = Field(max_digits=15, decimal_places=2)
bonificacion: Money = Field(max_digits=15, decimal_places=2)
saldo_final: Money = Field(max_digits=15, decimal_places=2)
source_filename: str
@@ -356,20 +369,20 @@ class BalanceOverride(SQLModel, table=True):
id: Optional[int] = Field(default=None, primary_key=True)
year: int
month: int
override_balance: float
override_balance: Money = Field(max_digits=15, decimal_places=2)
created_at: datetime = Field(default_factory=utcnow)
updated_at: datetime = Field(default_factory=utcnow)
class BalanceOverrideCreate(SQLModel):
override_balance: float
override_balance: Money
class BalanceOverrideRead(SQLModel):
id: int
year: int
month: int
override_balance: float
override_balance: Money
updated_at: datetime
@@ -379,8 +392,8 @@ class BalanceOverrideRead(SQLModel):
class SavingsAccrualBase(SQLModel):
year: int
month: int
memp_amount: float = 200000.0
mpat_amount: float = 200000.0
memp_amount: Money = Field(default=Decimal("200000"), max_digits=15, decimal_places=2)
mpat_amount: Money = Field(default=Decimal("200000"), max_digits=15, decimal_places=2)
trigger_transaction_id: Optional[int] = None
notes: Optional[str] = None
@@ -401,8 +414,8 @@ class SavingsAccrualRead(SavingsAccrualBase):
class SavingsAccrualUpdate(SQLModel):
memp_amount: Optional[float] = None
mpat_amount: Optional[float] = None
memp_amount: Optional[Money] = None
mpat_amount: Optional[Money] = None
notes: Optional[str] = None
@@ -418,10 +431,10 @@ class MunicipalReceiptBase(SQLModel):
holder_name: str
holder_cedula: str
holder_address: str
subtotal: float
interests: float
iva: float
total: float
subtotal: Money = Field(max_digits=15, decimal_places=2)
interests: Money = Field(max_digits=15, decimal_places=2)
iva: Money = Field(max_digits=15, decimal_places=2)
total: Money = Field(max_digits=15, decimal_places=2)
raw_charges: list[dict] = Field(
default_factory=list,
sa_column=Column(JSON, nullable=False, server_default="[]"),
@@ -453,15 +466,17 @@ class MunicipalReceiptRead(MunicipalReceiptBase):
class WaterMeterReadingBase(SQLModel):
meter_id: str
period: str # "YYYY-MM"
reading_previous: float = 0
reading_current: float = 0
consumption_m3: float
agua_potable: float = 0
serv_ambientales: float = 0
alcant_sanitario: float = 0
iva: float = 0
reading_previous: Money = Field(default=Decimal("0"), max_digits=15, decimal_places=2)
reading_current: Money = Field(default=Decimal("0"), max_digits=15, decimal_places=2)
consumption_m3: Money = Field(max_digits=15, decimal_places=2)
agua_potable: Money = Field(default=Decimal("0"), max_digits=15, decimal_places=2)
serv_ambientales: Money = Field(default=Decimal("0"), max_digits=15, decimal_places=2)
alcant_sanitario: Money = Field(default=Decimal("0"), max_digits=15, decimal_places=2)
iva: Money = Field(default=Decimal("0"), max_digits=15, decimal_places=2)
is_historical: bool = False
receipt_id: Optional[int] = Field(default=None, foreign_key="municipalreceipt.id")
receipt_id: Optional[int] = Field(
default=None, foreign_key="municipalreceipt.id", ondelete="CASCADE"
)
class WaterMeterReading(WaterMeterReadingBase, table=True):

View File

@@ -31,7 +31,7 @@ def get_effective_amount(item: RecurringItem, month: int, year: int) -> float |
if freq == RecurringFrequency.MONTHLY:
if item.override_amounts and str(month) in item.override_amounts:
return float(item.override_amounts[str(month)])
return item.amount
return float(item.amount)
if freq == RecurringFrequency.WEEKLY:
# Count occurrences of the weekday in this month
@@ -39,14 +39,14 @@ def get_effective_amount(item: RecurringItem, month: int, year: int) -> float |
weekday = item.day_of_month if item.day_of_month is not None else 0
cal = calendar.monthcalendar(year, month)
count = sum(1 for week in cal if week[weekday] != 0)
return item.amount * count
return float(item.amount) * count
if freq == RecurringFrequency.QUARTERLY:
# Active in months 3, 6, 9, 12 by default
if month % 3 == 0:
if item.override_amounts and str(month) in item.override_amounts:
return float(item.override_amounts[str(month)])
return item.amount
return float(item.amount)
return None
if freq == RecurringFrequency.BIANNUAL:
@@ -54,12 +54,12 @@ def get_effective_amount(item: RecurringItem, month: int, year: int) -> float |
base = item.month_of_year or 1
second = base + 6 if base <= 6 else base - 6
if month in (base, second):
return item.amount
return float(item.amount)
return None
if freq == RecurringFrequency.YEARLY:
if month == (item.month_of_year or 12):
return item.amount
return float(item.amount)
return None
return None
@@ -124,115 +124,65 @@ def compute_actuals_by_source(
amount_crc = get_converted_amount_expr(session)
def _grouped(*where) -> dict[tuple, tuple[float, int]]:
"""{(source, type): (total_crc, count)} for matching transactions."""
rows = session.exec(
select(
Transaction.source,
Transaction.transaction_type,
func.coalesce(func.sum(amount_crc), 0),
func.count(),
)
.where(*where)
.group_by(Transaction.source, Transaction.transaction_type)
).all()
return {(s, t): (float(total), cnt) for s, t, total, cnt in rows}
# Three grouped queries replace the previous ten single-aggregate ones.
cc_now = _grouped(
Transaction.date >= cc_start,
Transaction.date < cc_end,
Transaction.source == TransactionSource.CREDIT_CARD,
Transaction.deferred_to_next_cycle == False, # noqa: E712
)
cc_deferred = _grouped(
Transaction.date >= prev_start,
Transaction.date < prev_end,
Transaction.source == TransactionSource.CREDIT_CARD,
Transaction.deferred_to_next_cycle == True, # noqa: E712
)
non_cc = _grouped(
Transaction.date >= cal_start,
Transaction.date < cal_end,
Transaction.source != TransactionSource.CREDIT_CARD,
)
results = {}
for source in TransactionSource:
if source == TransactionSource.CREDIT_CARD:
start, end = cc_start, cc_end
# Normal transactions in this cycle (not deferred)
compra_normal = session.exec(
select(func.coalesce(func.sum(amount_crc), 0)).where(
Transaction.date >= start,
Transaction.date < end,
Transaction.source == source,
Transaction.transaction_type == TransactionType.COMPRA,
Transaction.deferred_to_next_cycle == False, # noqa: E712
)
).one()
# Deferred from previous cycle
compra_deferred = session.exec(
select(func.coalesce(func.sum(amount_crc), 0)).where(
Transaction.date >= prev_start,
Transaction.date < prev_end,
Transaction.source == source,
Transaction.transaction_type == TransactionType.COMPRA,
Transaction.deferred_to_next_cycle == True, # noqa: E712
)
).one()
compra = float(compra_normal) + float(compra_deferred)
dev_normal = session.exec(
select(func.coalesce(func.sum(amount_crc), 0)).where(
Transaction.date >= start,
Transaction.date < end,
Transaction.source == source,
Transaction.transaction_type == TransactionType.DEVOLUCION,
Transaction.deferred_to_next_cycle == False, # noqa: E712
)
).one()
dev_deferred = session.exec(
select(func.coalesce(func.sum(amount_crc), 0)).where(
Transaction.date >= prev_start,
Transaction.date < prev_end,
Transaction.source == source,
Transaction.transaction_type == TransactionType.DEVOLUCION,
Transaction.deferred_to_next_cycle == True, # noqa: E712
)
).one()
devolucion = float(dev_normal) + float(dev_deferred)
count_normal = session.exec(
select(func.count()).where(
Transaction.date >= start,
Transaction.date < end,
Transaction.source == source,
col(Transaction.transaction_type).notin_(INCOME_TYPES),
Transaction.deferred_to_next_cycle == False, # noqa: E712
)
).one()
count_deferred = session.exec(
select(func.count()).where(
Transaction.date >= prev_start,
Transaction.date < prev_end,
Transaction.source == source,
col(Transaction.transaction_type).notin_(INCOME_TYPES),
Transaction.deferred_to_next_cycle == True, # noqa: E712
)
).one()
count = count_normal + count_deferred
results[source.value] = {
"source": source.value,
"total_compra": compra,
"total_devolucion": devolucion,
"net": compra - devolucion,
"count": count,
}
buckets = (cc_now, cc_deferred)
else:
# Cash / Transfer: calendar month, no deferred logic
compra = session.exec(
select(func.coalesce(func.sum(amount_crc), 0)).where(
Transaction.date >= cal_start,
Transaction.date < cal_end,
Transaction.source == source,
Transaction.transaction_type == TransactionType.COMPRA,
)
).one()
devolucion = session.exec(
select(func.coalesce(func.sum(amount_crc), 0)).where(
Transaction.date >= cal_start,
Transaction.date < cal_end,
Transaction.source == source,
Transaction.transaction_type == TransactionType.DEVOLUCION,
)
).one()
count = session.exec(
select(func.count()).where(
Transaction.date >= cal_start,
Transaction.date < cal_end,
Transaction.source == source,
col(Transaction.transaction_type).notin_(INCOME_TYPES),
)
).one()
compra_val = float(compra)
devolucion_val = float(devolucion)
results[source.value] = {
"source": source.value,
"total_compra": compra_val,
"total_devolucion": devolucion_val,
"net": compra_val - devolucion_val,
"count": count,
}
buckets = (non_cc,)
compra = sum(
b.get((source, TransactionType.COMPRA), (0.0, 0))[0] for b in buckets
)
devolucion = sum(
b.get((source, TransactionType.DEVOLUCION), (0.0, 0))[0]
for b in buckets
)
count = sum(
cnt
for b in buckets
for (s, t), (_total, cnt) in b.items()
if s == source and t not in INCOME_TYPES
)
results[source.value] = {
"source": source.value,
"total_compra": compra,
"total_devolucion": devolucion,
"net": compra - devolucion,
"count": count,
}
return results
@@ -380,18 +330,17 @@ def compute_cc_by_category(
).all()
)
# Resolve category names
# Resolve category names in one query instead of one per category
from app.models.models import Category
names = {
c.id: c.name for c in session.exec(select(Category)).all()
}
result = []
for cat_id, amount in totals.items():
if amount <= 0:
continue
if cat_id is not None:
cat = session.get(Category, cat_id)
name = cat.name if cat else "Sin categoría"
else:
name = "Sin categoría"
name = names.get(cat_id, "Sin categoría") if cat_id is not None else "Sin categoría"
result.append({"category_name": name, "amount": round(amount, 2)})
return sorted(result, key=lambda x: x["amount"], reverse=True)
@@ -561,13 +510,13 @@ def _get_december_cumulative(session: Session, year: int) -> float:
)
).first()
if override:
return override.override_balance
return float(override.override_balance)
# Compute the full year to get December's cumulative
overrides = session.exec(
select(BalanceOverride).where(BalanceOverride.year == year)
).all()
override_map = {o.month: o.override_balance for o in overrides}
override_map = {o.month: float(o.override_balance) for o in overrides}
cumulative = 0.0
if year > FRESH_START_YEAR:
@@ -591,7 +540,7 @@ def compute_yearly_projection_with_cumulative(
overrides = session.exec(
select(BalanceOverride).where(BalanceOverride.year == year)
).all()
override_map = {o.month: o.override_balance for o in overrides}
override_map = {o.month: float(o.override_balance) for o in overrides}
# Determine January carryover
if year <= FRESH_START_YEAR:

View File

@@ -61,8 +61,8 @@ def _fetch_bccr_rate(indicator: int, date_str: str) -> float | None:
for datos in root.iter():
if datos.tag.endswith("NUM_VALOR"):
return float(datos.text.strip().replace(",", "."))
except Exception:
pass
except Exception as e:
logger.warning("BCCR rate fetch failed (indicator %s): %s", indicator, e)
return None
@@ -91,8 +91,8 @@ def _fetch_exchangerate_api() -> tuple[float, float] | None:
crc = data["rates"].get("CRC")
if crc:
return _mid_to_buy_sell(float(crc))
except Exception:
pass
except Exception as e:
logger.warning("ExchangeRate-API fetch failed: %s", e)
return None
@@ -106,7 +106,8 @@ def _fetch_currency_api() -> tuple[float, float] | None:
crc = data.get("usd", {}).get("crc")
if crc:
return _mid_to_buy_sell(float(crc))
except Exception:
except Exception as e:
logger.warning("currency-api fetch failed (%s): %s", url, e)
continue
return None
@@ -120,8 +121,8 @@ def _fetch_floatrates() -> tuple[float, float] | None:
crc_data = data.get("crc")
if crc_data and "rate" in crc_data:
return _mid_to_buy_sell(float(crc_data["rate"]))
except Exception:
pass
except Exception as e:
logger.warning("FloatRates fetch failed: %s", e)
return None
@@ -199,8 +200,8 @@ def _fetch_fiat_crc_mid(code: str) -> float | None:
x = data["rates"].get(code)
if crc and x:
return float(crc) / float(x)
except Exception:
pass
except Exception as e:
logger.warning("%s/CRC fiat rate fetch failed: %s", code, e)
return None
@@ -220,8 +221,8 @@ def _fetch_crypto_crc(code: str) -> float | None:
price = data.get(coin_id, {}).get("crc")
if price:
return float(price)
except Exception:
pass
except Exception as e:
logger.warning("CoinGecko %s/CRC fetch failed: %s", code, e)
return None
@@ -255,7 +256,8 @@ def get_crc_multipliers(session: Session) -> dict[str, float]:
usd_rate = get_current_rate(session)
if usd_rate:
multipliers["USD"] = usd_rate.sell_rate
# sell_rate is Decimal when loaded from the DB; keep multipliers float
multipliers["USD"] = float(usd_rate.sell_rate)
for code in (c.value for c in Currency):
if code in multipliers:

View File

@@ -1,3 +1,5 @@
from decimal import Decimal
from sqlmodel import Session, select
from app.models.models import (
@@ -8,8 +10,8 @@ from app.models.models import (
Transaction,
)
MEMP_MONTHLY = 200000.0
MPAT_MONTHLY = 200000.0
MEMP_MONTHLY = Decimal("200000")
MPAT_MONTHLY = Decimal("200000")
def _get_savings_account(session: Session, bank: Bank) -> Account | None:

22
backend/app/uploads.py Normal file
View File

@@ -0,0 +1,22 @@
from fastapi import UploadFile
MAX_UPLOAD_BYTES = 10 * 1024 * 1024 # 10 MB — statements are a few hundred KB
async def read_pdf_upload(
file: UploadFile, max_bytes: int = MAX_UPLOAD_BYTES
) -> bytes:
"""Read an uploaded PDF with a size cap and magic-byte check.
Raises ValueError with a user-readable, filename-prefixed message —
upload endpoints surface these in their per-file `errors` list.
"""
filename = file.filename or "unknown.pdf"
data = await file.read(max_bytes + 1)
if len(data) > max_bytes:
raise ValueError(
f"{filename}: file exceeds the {max_bytes // (1024 * 1024)} MB limit"
)
if not data.startswith(b"%PDF"):
raise ValueError(f"{filename}: not a PDF file")
return data

View File

@@ -0,0 +1,151 @@
"""Agent tool coverage: session binding, JSON-safe output (no Decimal leaks),
and the batched queries introduced in Phase 2."""
import json
from datetime import date, datetime
from decimal import Decimal
import pytest
from app.agent import tools
from app.agent.tools import reset_session, set_session
from app.models.models import (
Account,
AccountType,
Bank,
Currency,
MunicipalReceipt,
PensionSnapshot,
Transaction,
WaterMeterReading,
)
@pytest.fixture()
def bound_session(session):
token = set_session(session)
yield session
reset_session(token)
def make_snapshot(fund: Bank, start: date, end: date, saldo: str) -> PensionSnapshot:
zero = Decimal("0")
return PensionSnapshot(
fund=fund,
contract_number="C-1",
period_start=start,
period_end=end,
saldo_anterior=zero,
aportes=zero,
rendimientos=zero,
retiros=zero,
traslados=zero,
comision=zero,
correccion=zero,
bonificacion=zero,
saldo_final=Decimal(saldo),
source_filename="t.pdf",
)
def make_receipt(period: str) -> MunicipalReceipt:
zero = Decimal("0")
return MunicipalReceipt(
receipt_date=date(2026, 4, 1),
due_date=date(2026, 4, 15),
period=period,
account="A1",
finca="F1",
holder_name="x",
holder_cedula="x",
holder_address="x",
subtotal=Decimal("100"),
interests=zero,
iva=Decimal("13"),
total=Decimal("113"),
source_filename="r.pdf",
)
def test_unbound_session_raises(session):
with pytest.raises(RuntimeError, match="not bound to agent context"):
tools.get_accounts()
def test_accounts_output_is_json_safe(bound_session):
bound_session.add(
Account(
bank=Bank.BAC,
currency=Currency.CRC,
label="Main",
balance=Decimal("1234.56"),
account_type=AccountType.BANK,
)
)
bound_session.commit()
out = tools.get_accounts()
json.dumps(out) # would raise on Decimal leakage
assert out[0]["balance"] == 1234.56
def test_pension_latest_only_per_fund(bound_session):
bound_session.add(
make_snapshot(Bank.FCL, date(2026, 1, 1), date(2026, 1, 31), "100")
)
bound_session.add(
make_snapshot(Bank.FCL, date(2026, 2, 1), date(2026, 2, 28), "200")
)
bound_session.add(
make_snapshot(Bank.ROP, date(2026, 1, 1), date(2026, 1, 31), "900")
)
bound_session.commit()
out = tools.get_pension_snapshots(latest_only=True)
json.dumps(out)
assert len(out) == 2
by_fund = {o["fund"]: o for o in out}
assert by_fund["FCL"]["saldo_final"] == 200.0 # latest period wins
assert by_fund["ROP"]["saldo_final"] == 900.0
full = tools.get_pension_snapshots(latest_only=False)
assert len(full) == 3
def test_municipal_receipts_batched_consumption(bound_session):
r1, r2 = make_receipt("2026-03"), make_receipt("2026-04")
r2.account = "A2"
bound_session.add(r1)
bound_session.add(r2)
bound_session.commit()
bound_session.refresh(r1)
bound_session.refresh(r2)
for rid, m3 in ((r1.id, "10.5"), (r1.id, "4.5"), (r2.id, "7")):
bound_session.add(
WaterMeterReading(
meter_id="7335",
period=f"p{m3}",
consumption_m3=Decimal(m3),
receipt_id=rid,
)
)
bound_session.commit()
out = tools.get_municipal_receipts()
json.dumps(out)
by_account = {o["account"]: o for o in out}
assert by_account["A1"]["water_consumption_m3"] == 15.0
assert by_account["A2"]["water_consumption_m3"] == 7.0
def test_recent_transactions_json_safe(bound_session):
bound_session.add(
Transaction(
amount=Decimal("42.42"),
merchant="M",
date=datetime(2026, 4, 1),
)
)
bound_session.commit()
out = tools.get_recent_transactions()
json.dumps(out)
assert out[0]["amount"] == 42.42

View File

@@ -0,0 +1,67 @@
"""Pin the Money serialization contract (Phase 2 Decimal migration, BE-02):
NUMERIC storage, but JSON emits plain numbers so the frontend's `number`
types keep working."""
import json
from datetime import datetime
from decimal import Decimal
from sqlmodel import select
from app.models.models import (
Account,
AccountRead,
Bank,
Currency,
Transaction,
TransactionRead,
)
def test_money_columns_are_numeric():
assert str(Transaction.__table__.c.amount.type) == "NUMERIC(15, 2)"
assert str(Account.__table__.c.balance.type) == "NUMERIC(15, 2)"
def test_money_serializes_as_json_number():
read = TransactionRead(
id=1,
amount=Decimal("25000.50"),
merchant="X",
date=datetime(2026, 4, 1),
created_at=datetime(2026, 4, 1),
)
dumped = read.model_dump(mode="json")
assert dumped["amount"] == 25000.5
assert isinstance(dumped["amount"], float)
# and the whole payload is json.dumps-able (no Decimal leaks)
json.dumps(dumped)
def test_money_accepts_float_input():
# n8n and the SPA POST plain JSON numbers; they must coerce to Decimal.
read = AccountRead(
id=1,
bank=Bank.BAC,
currency=Currency.CRC,
label="x",
balance=1234.56,
updated_at=datetime(2026, 4, 1),
)
assert read.balance == Decimal("1234.56")
def test_decimal_roundtrip_through_db(session):
session.add(
Transaction(
amount=Decimal("9999.99"),
merchant="ROUNDTRIP",
date=datetime(2026, 4, 1),
)
)
session.commit()
row = session.exec(
select(Transaction).where(Transaction.merchant == "ROUNDTRIP")
).one()
assert isinstance(row.amount, Decimal)
assert row.amount == Decimal("9999.99")

View File

@@ -0,0 +1,41 @@
"""Upload validation: size cap and PDF magic-byte check (SEC-06)."""
import asyncio
from io import BytesIO
import pytest
from fastapi import UploadFile
from app.uploads import read_pdf_upload
def make_upload(data: bytes, filename: str = "test.pdf") -> UploadFile:
return UploadFile(file=BytesIO(data), filename=filename)
def test_valid_pdf_accepted():
data = b"%PDF-1.7 fake content"
out = asyncio.run(read_pdf_upload(make_upload(data)))
assert out == data
def test_non_pdf_rejected_with_filename():
with pytest.raises(ValueError, match=r"evil\.pdf: not a PDF"):
asyncio.run(read_pdf_upload(make_upload(b"MZ\x90\x00binary", "evil.pdf")))
def test_oversized_rejected():
big = b"%PDF" + b"x" * 100
with pytest.raises(ValueError, match="exceeds the 0 MB limit"):
asyncio.run(read_pdf_upload(make_upload(big), max_bytes=50))
def test_exact_limit_accepted():
data = b"%PDF" + b"x" * 46 # exactly 50 bytes
out = asyncio.run(read_pdf_upload(make_upload(data), max_bytes=50))
assert len(out) == 50
def test_empty_file_rejected():
with pytest.raises(ValueError, match="not a PDF"):
asyncio.run(read_pdf_upload(make_upload(b"")))

View File

@@ -59,7 +59,44 @@ Completed 2026-06-09.
---
## Phase 2 — Backend Correctness & Robustness ⏳ NOT STARTED
## Phase 2 — Backend Correctness & Robustness ✅ DONE
Completed 2026-06-10.
| Plan item | Status | Commit |
|---|---|---|
| 2.1 Money → Decimal/NUMERIC (29 columns, BE-02) | ✅ | `74886fb` |
| 2.2 Upload hardening: 10 MB cap, %PDF magic, to_thread (SEC-06, BE-14) | ✅ | (uploads) |
| 2.3 FK delete rules, grouped queries (10→3), N+1 fixes, tiebreakers | ✅ | `3cdd7d9` |
| 2.4 Error visibility: per-source rate logging, freshness, clear agent errors | ✅ | `82f10a5` |
| 2.5 Agent cleanup: real multipliers, year bounds, offsets, read-only policy | ✅ | `82f10a5` |
| 2.6 Consistency: enums, keyword status codes, SimpleCookie, import hygiene | ✅ | `440da3e` |
**Key decisions:**
- **Decimal strategy:** `Money = Annotated[Decimal, PlainSerializer(float, when_used="json")]`
— storage is exact `NUMERIC(15,2)` (rates 15,6) while JSON keeps emitting plain
numbers, so the frontend contract is unchanged (pinned by
`test_models_serialization.py`). Services coerce to float at their computation
boundaries. Migration `7f567c8bafdb` verified on dev: per-row values and table
sums byte-identical across the type change.
- **FK rules** (`7884505b16b3`): transactions/recurring items `SET NULL` on category
delete — this also fixes the 500 that `DELETE /categories` raised for in-use
categories; water readings `CASCADE`. Verified with a rolled-back probe on dev.
- **Agent net worth** no longer guesses (hardcoded 600 CRC / 1.08 EUR removed): it
converts via `get_crc_multipliers` and reports unconvertible accounts explicitly.
- **Read-only tool policy** documented in `agent/tools.py` (SEC-09): ingested email
text is a prompt-injection surface; any future write tool needs UI confirmation.
**Review findings closed as overstated/false during implementation:**
- BE-09 (temp-file leak): parsers already use context-managed temp files.
- BE-10 (per-file commits): the pension batch already commits atomically.
- ARCH-08 (swallowed cancellation): `CancelledError` is a `BaseException` since
Python 3.8 — `except Exception` never caught it.
- ARCH-03 (filter precedence): `if cycle / elif dates` is already explicit.
- BE-22 (upsert race): accepted as-is — single writer (daily n8n cron), batch-
scoped commit; ON CONFLICT complexity not warranted.
Tests: 55 → **69** (new: model serialization, agent tools, upload validation).
## Phase 3 — Frontend Data Layer & Feedback ⏳ NOT STARTED