Skip to content

Feat: Translation Memory#30

Merged
TenzDelek merged 6 commits intomainfrom
tm
Apr 1, 2026
Merged

Feat: Translation Memory#30
TenzDelek merged 6 commits intomainfrom
tm

Conversation

@TenzDelek
Copy link
Copy Markdown
Member

@TenzDelek TenzDelek commented Apr 1, 2026

Summary

Add Translation Memory (TM) system to optimize translation workflows by caching and reusing previous translations. The system provides both exact matching for instant retrieval and fuzzy matching using PostgreSQL's trigram similarity to surface relevant prior translations, significantly reducing LLM API calls and improving response times for repeated content.

{
  "results": [
    {
      "output_text": "ཁྱེད་རང་ག་དེ་འདུག",
      "from_memory": true,
      "fuzzy_matches": []
    }
  ],
  "metadata": {
    "initialized_at": "2026-04-01T06:25:29.735603+00:00",
    "total_batches": 0,
    "completed_at": "2026-04-01T06:25:29.735603+00:00",
    "total_processing_time": 0
  },
  "errors": []
}

@TenzDelek TenzDelek changed the title Tm Feat: Translation Memory Apr 1, 2026
if target_language:
with SessionLocal() as db_session:
for idx, source_text in enumerate(prompt):
exact = find_exact_match(
Copy link
Copy Markdown
Member Author

@TenzDelek TenzDelek Apr 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

first try to find exact match with the input, if matched ignore the llm step

tm_hits[idx] = exact.target_text
continue

rows = find_fuzzy_matches(
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if fails then we fuzzy match with the input

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here the input still go under llm translation

assistant_id, prompt, target_language
)

if not texts_for_llm:
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if true, that means exact match successes so we return it without the need to call the llm

).first()


def find_fuzzy_matches(
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fuzzy match logic generated via opus

Copy link
Copy Markdown
Member Author

@TenzDelek TenzDelek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

self reviewed

@TenzDelek TenzDelek marked this pull request as ready for review April 1, 2026 05:59
@TenzDelek TenzDelek merged commit b64c816 into main Apr 1, 2026
2 checks passed
@TenzDelek TenzDelek deleted the tm branch April 1, 2026 06:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant