How do i use environment variables to make a indexer? I couldn’t get it to work

Repeatable config options are not supported from env vars. In this case you have to mount a config file to your container. But, if you want to track local files from docker, you have to mount those files/directories anyway. I suggest to use the standalone binary release if you’d like to track local files, it is way more convenient.

can i connect it to other services like paperless, or would i need to manually import files?

Hister supports importing data from a few services, but paperless isn’t supported yet. More details: https://hister.org/docs/import

Is it better to run it in docker or e.g. a LXC in proxmox? If i want to index files, it seems it needs the config.yml file

Docker is perfectly fine. Every settings option from the config file can specified using environment variables. The syntax is HISTER__[SECTION]__[OPTION]=[VALUE].

After just playing with it for a day, the disk usage is 150 MB

Probably most of the disk space is occupied by the Hister binary which contains all the N-grams required to identify ~30 languages. The index should be much smaller.

I figure I should use postgres instead of sqlite.

SQLite is more than enough for personal use, but if you prefer to use postgres, just specify the standard DSN formatted connection data to the server.database config option: https://hister.org/docs/configuration#database-backends

how to set that up, preferably with docker

Use the HISTER__SERVER__DATABASE="host=localhost user=hister password=hister dbname=hister port=5432" environment variable.

I am not familiar with pgvector

Hister automatically creates the database model and handles the migrations if required.

Currently it is not planned. Hister guarantees that non of your data/query/metadata leaves the service if you use it. As I see, this is a more valuable and unique feature than having an integrated metasearch. There are already great metasearch solutions and Hister provides an easy fallback to search providers, so in my opinion this direction would be more of a sacrifice than an improvement.

The results are standard search results, if you click on the title, it opens the original URL, just like in any other regular search engine. You can even close the preview pane if you are not interested in it.

However, dismissing it as “a multiple screens long AI prompt” does not only not answer the question, it comes off as abrasive.

I think it is more abrasive to copy/paste a poorly formatted LLM output instead of taking the time and summarizing it to a few sentences just as you did in your previous post.

Is there a TTL / max database size per user setting?

As I wrote, you can simply automate deletion by document age. Schedule a delete event on each day with the desired retention time defined as a filter expression. Database size limit isn’t available yet.

Additionally, is the other parenthetical information materially correct? If not, which points [1 thru to 7] are wrong?

No, sorry, I don’t have time to correct a copy of a multiple screens long AI prompt.

I would like to further recommend Hister but your documentation is somewhat confusing at first blush.

Which parts are confusing?

The summary has numerous inaccuracies. Most importantly: it is pretty easy to delete content by topic or age. The hister delete command can accept a search query to remove only matched documents. The same is true on the web UI “actions -> remove all matching documents”. You can quickly filter by age, simply query updated:>365d. Combine it with URLs, labels, domains or phrases.

I’ve started creating smaller, pre-fetched, thematic datasets: https://hister.org/datasets . It is still very rudimentary, but the concept is the same. Both datasets and improvement ideas are appreciated.

It isn’t possible to setup hister as my default search engine in browser with automated fallback to normal Meta-search with whatever engine?

It can be added as a default search engine and it provides a link and a hotkey to open the search query with an external search engine, however I have a different workflow: https://hister.org/posts/how-i-use-hister

And I use login credentials maybe a faster way like passkey possible in the future?

That’s a good idea, thanks for the suggestion.

Though maybe the resource folder can be ignored and the html file can already be imported?

It depends. Hister always requires a unique URL for each document. SingleFile snapshots include the original URL of the document as a meta HTML element. I’m not sure if the built-in page saver provides URL information.

I’d appreciate it, thanks. No special requirements. Providing sensible defaults and explaining usage/potential customization options would be great.

The storage requirement is around 100KB/page on average.

Memory usage can exceed 1GB momentarily for searches when using language detection and multi-language indexes (it is the default config). Without language detection Hister has a much smaller memory footprint (~30MB default with ~100-150MB peaks).

Exactly, the echo chamber phenomenon is mostly problematic for “discovery type” searches while Hister is mainly for “recall type” search.

Implementing federated search/index sharing could be a partial solution to this issue in the long run.

Oh damn this could replace the bookmakers.

The inspiration for Hister was a bookmarking app, but I realized that I always forget to manually trigger the bookmarking and I miss so many great resources.

Would it be possible to attach an HTML file from like single file

Currently you can import SingleFile HTMLs using the hister import file command, but no further integrations are implemented yet. Although, rendering the exact SinglePage file as a preview can be added relatively quickly. It would be also nice to accept files directly from the SingleFile extension.

Thanks for the good suggestion, I’ve added it to my TODO! =]

Federated search is the direction I’d like to go when the core is mature enough.

I’m still trying to figure out the best approach to make the federation secure (no accidental private/confidential data leak) and easy to use. Related conversations: https://github.com/asciimoo/hister/discussions/432 & https://github.com/asciimoo/hister/issues/387 . I’d appreciate help to figure out an optimal solution.

Thanks for letting me know. I’ve updated the post.