Skip to content

Add Ollama provider and streaming translation output support#461

Open
Shungy wants to merge 9 commits into
dialect-app:mainfrom
Shungy:ollama
Open

Add Ollama provider and streaming translation output support#461
Shungy wants to merge 9 commits into
dialect-app:mainfrom
Shungy:ollama

Conversation

@Shungy
Copy link
Copy Markdown

@Shungy Shungy commented Mar 29, 2026

This PR adds Ollama as a provider and enables streaming translation response.

Ollama.streaming.translation.in.Dialect.mp4

Using Ollama API one can either add local or remote instances for LLM based translations. LLMs provide versatility the traditional translation engines cannot provide. For example, the translation above is using translategemma:12b locally, and it is fairly correct for an archaic piece of text from 17th century. Google Translate completely fails for such texts. Another strength is that it does not require specifying the source language nor it tries to explicitly detect it, because it translates based on understanding and meaning regardless of the classification of the source language. This makes its usage very simple as one does not need to modify the source language constantly. That being said, LLM-based translations can suffer from hallucinations, leading to outright incorrect translations due to prior bias, missing knowledge, or knowledge conflict. All in all, this method has its strengths and weaknesses, and it can be very useful if one is aware of its limitations.

In order to have a proper support for the Ollama provider, the PR also adds ENGINES and STREAMING provider features. The engines is simply an additional setting to specify which LLM model to use. I have named it "engine" for it to be more universal, as other non-LLM APIs might offer this setting. The streaming feature is for continuous translation responses. Since LLMs do not wait for the entire response to be complete, and instead output results token by token, it made sense to add this feature. Other than it is nice to see the output as it is being translated, it is also a better UX because LLM generation can take a long time depending on the model and the hardware being used. One feature that is missing is the inability to stop the translations, which I will either add to this PR or create a separate PR depending on the feedback.

The Ollama provider also has bunch of supported languages listed by default. Since we cannot query an LLM on the languages it supports, I believed the listed languages is a good amount. The more languages listed would offer a bad experience due to required scrolling, and would be more likely to misinform users on which languages a given model supports. The less languages listed would artificially limit the users on which target language they can translate to. I think this is an acceptable compromise without having to rework the UI.

Shungy added 9 commits March 27, 2026 21:36
do not reinvent the wheel
If provider has streaming feature, `window.py::_on_translation` now
chooses `stream_translate` over `translate`. Hence it is redundant to
have a separate `translate` function in the provider.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant