It replaced both Filebeat and Logstash for us with a single binary that actually has sane resource usage (no more JVM nightmares). VRL turned out to be way more powerful than we could imagine - we do all our log parsing, metadata enrichment, and routing to different ClickHouse tables in one place. The agent/aggregator topology with disk buffering is pretty dope.
Genuinely one of my favorite pieces of infra software. Good luck with Tero.
I've landed on a similar philosophy but with a slightly different approach to orchestration. Instead of managing everything interactively, I built a lightweight bash-based deployment system that uses rsync + docker compose across multiple machines.
The structure is dead simple: `machines/<hostname>/stacks/<service>/` with a `config.sh` per machine defining SSH settings and optional pre/post deploy hooks. One command syncs files and runs `docker compose up -d`.
I could see Claude Code being useful for debugging compose files or generating new stack configs, but having the deployment itself be a single `./deploy.sh homeserver media` keeps the feedback loop tight and auditable.
It's simple enough and I had some prior experience with it, so I merely have some variables, roles that render a docker-compose.yml.j2 template and boom. It all works, I have easy access to secrets, shared variables among stacks and run it with a simple `ansible-playbook` call.
If I forget/don't know the Ansible modules, Claude or their docs are really easy to use.
Every time I went down a bash script route I felt like I was re-inventing something like Ansible.
Garage is really good for core S3, the only thing I ran into was it didn't support object tagging. It could be considered maybe a more esoteric corner of the S3 api, but minio does support it. Especially if you're just mapping for a test api, object tagging is most likely an unneeded feature anyway.
I also quite like Localsend to quickly share files on the same network. Used to send photos/files to myself on Whatsapp/Telegram before but after discovering Localsend, stopped!
I've been working on LogChef (https://logchef.app) - a specialized log analytics UI for ClickHouse that focuses on powerful querying and exploration without the complexity of full observability platforms.
The core idea is to leverage ClickHouse's incredible columnar performance for log analytics while providing a schema-agnostic interface that works with any log table structure. It supports both simple search syntax for quick queries and full ClickHouse SQL for complex analytics. Also it has proper RBAC: Team-based access controls for multi-tenant environments.
Off late I have also added some AI features:
- AI-powered SQL generation - write queries in natural language
- MCP (Model Context Protocol) server integration for AI assistants to query your logs
It's open source (AGPLv3) and deliberately doesn't handle log collection - instead it integrates with existing tools like Vector, Fluentd, or OpenTelemetry Collector. The roadmap includes REST APIs, client libraries, visualizations, and alerting.
Built with Go + Vue.js + TypeScript. Currently handles millions of log entries daily in production environments at my org. The deployment is just a single binary deployment with a SQLite DB.
It's great to see Typst getting more visibility. We migrated a real-world workload to it a couple of years back: generating and e-mailing 1.5M+ PDFs daily at my org, Zerodha.
Our previous pipeline was LaTeX-based (first pdflatex, then lualatex), but we were constantly fighting cryptic memory errors on large documents and huge Docker image sizes that slowed down boot times of our ephemeral workers.
Switching to Typst was a massive win for us. The single static binary resulted in tiny images and faster boot times. More importantly, the performance gains were huge. Overall compile times were ~3–4× faster than LaTeX. On really large documents (2000+ pages, mostly tables), Typst compiles in ~1 minute vs. ~18 minutes with lualatex.
Beyond performance, the better developer experience and good error messages was a nice bonus too.
We wrote a detailed post about the entire architecture - from the job orchestration with Nomad to the S3 optimizations and the Typst migration in particular. If you're curious, you can read it here: https://zerodha.tech/blog/1-5-million-pdfs-in-25-minutes
I am using a LaTeX pipeline for creating text, invoices, forms, etc. from snippets stored in a database. It was quite complicated to set up, but the results are very satisfying. In principle, I would like an easier markup language, but am afraid to wast my time trying out something new, just to run into limitations after awhile.
Here are my most important requirements: Multiple columns configurable for meaningful column and page breaks (minimum number of lines in a new column or page). Reliable automatic hyphenation for multiple languages (at least English and German, perhaps French, Italian and Spanish in the future). Pictures in columns with dynamic placing according to the available space. Complex tables and forms with sensible automatic page breaks. Background images. Different areas on the page which are treated like mini-pages. -- Is Typst capable of all that?
Now, this isn't really an alternative, but I can recommend using pandoc instead of pure LaTex. Personally, I think text blocks are much easier to handle and read in markdown and pandoc allows you to use in-line LaTex wherever markdown is insufficient, without making basic text styling unnecessarily verbose. In my opinion, the result is a better/friendlier LaTex, or limitless markdown respectively. Especially for writing papers and such, you can always start actually writing in basic markdown, which is trivial and almost-WYSIWYG, and later worry about presentation. In my experience, with pure LaTex you easily get lost in sidequests and it's much harder to get into a writing flow and find your way around your own words in the editor.
I wrote my dissertation this way. As long as you’re already comfortable with latex, it’s amazing. Huge time saver not to have to write latex formatting when you don’t have to, but an equally huge time saver to be able to control formatting when you need it.
It's a bit of a weird thing, really. I think, I have not seen any comprehensive, basic tutorial on the matter. I presume a lot of people who've become victim of the LaTex bug, but dislike its overhead, independently found bliss in this pandoc compromise. It's definitely a bit of coping with LaTex Stockholm syndrome, you probably need a specific set of experiences, know some struggle to fully appreciate it.
If anyone here hasn't messed with it, but thinks "I wonder, if LaTex could ruin my life, too", I suggest to start learning pandoc first (it can do various markup conversions, it's useful by itself). Get familiar with pandoc metadata headers on a RTFM basis. A LaTex intro is all you need after that, if you are up for a bit of trial and error. Or learn LaTex first, but not both at the same time, as the layered processing involved is a bit confusing (e.g. pandoc showing pdflatex errors). In any case, I strongly advice to bite the bullet and install the full LaTex package of your distro (no idea about macOS, or windows), as fighting dependencies is suuuper annoying. However, the trade-off is it's truly a behemoth, consequently frequent upgrades for a lot of packages you don't ever use. (I configured dnf to download upgrades automatically, so I can do a quick `dnf upgrade --cacheonly` when opportune.)
Speaking of... I wish pandoc could do Tex package management and dependency resolution.
I ended up just downloading the texlive image and letting it go slightly stale while I wrote. Latex plays poorly with OS package managers and I decided I preferred to be out of date.
I made each chapter a separate file and compiled each one to latex. I had a top level latex file that did an \include{} for each chapter, and a big old Makefile to tie everything together, including figure generation.
All this, and it was still easier than writing every latex command by hand.
If you're going to do this, might as well go with AsciiDoc instead. It's vastly superior as a syntax. Markdown breaks down at the mere mention of something as trivial as a nested list item containing a table of blockquotes.
Not that it couldn’t be trivial in the abstract, but I’m struggling to imagine a use for a nested list item containing a table of blockquotes. It doesn’t seem at all surprising that a tool wouldn’t anticipate that.
Absolutely! The writers are generally not highly technically experienced, but have to produce highly technical documents that need to be regularly reviewed and changed for years at a time.
If you're going to do this, why not generate Pandoc ASTs directly? You can do so from a number of languages and they support (by definition) a superset of any given markup's features, with blocks to call out directly for things you can only do in Latex.
I assume the original question is asking about programmatic document generation, in which case working with a real AST is probably also a productivity and reliability win as well.
I’ve worked places that will regularly send generated PDFs of things like statistics or small reports or other generic boring business stuff every week or month.
It’s always been some combination of MJML, inline DIY HTML, or PDFs generated with Puppeteer etc.
Same, used to work at a place that used to use ghostscript + chromedriver and went to Weasyprint.
Weasyprint is quite wonderful, fast, and you can file upstream tickets and get good clarity on what's going on.
And you're writing your templates in CSS + HTML, which generally flows well.
At the end of the day the problems are rarely "CSS + HTML vs Latex vs Typst", but more just about making decisions about how you want something to be laid out when you have dynamic input.
"What should happen when I have a huge blob of text here" is often a non-trivial question. "What should I do on a page break here" is a non-trivial question. And they often involve having to, at the high level, make decisions. People whine about CSS, sometimes you just gotta read the spec and figure things out (rather than stop at your intuition for how certain fields work).
Nice thing about CSS + HTML is you can open up your output in a browser and futz about with it very easily.
If you're compiling millions of documents, many with thousands of pages, you probably need something very universal. LaTeX is boring tech, off the shelf, ready to use. It might take some work to figure out the initial setup with regards to templating and everything else, but after that, you can be generally pretty content that it will handle most things you throw at it just fine.
I am currently preparing to switch to DITA. The learning curve is steeper at the beginning, but I find the overall concept of topic-oriented, information-typed authoring with content reuse very attractive.
Some people might say that writing in XML is annoying, but it isn't if you have a decent XML editor. In my case, it is Emacs nXML mode. Customisation is possible with DITA-OT [1] and plugins, and yes, it is also based on XSLT. Overall, I think DITA is an industry-proven XML powerhouse. It may be boring, but it has huge potential for anyone with advanced documentation requirements.
Your original comment before editing complained about using pdflatex as if it was not part of the pandoc toolchain. It was not about pandoc being universal.
It's an open-source (AGPLv3), purpose-built log analytics UI specifically for ClickHouse. I've been using ClickHouse for logs and love its power, but found the existing UIs either too heavy, too focused on ingestion (which we already have covered with tools like Vector/Promtail/Fluentbit/Logstash etc), or not ClickHouse-native enough.
Logchef aims to be a lightweight, powerful log explorer that sits on top of your existing ClickHouse setup.
Key things:
- Schema-agnostic: Works with your existing ClickHouse log tables. No need to change how you store things.
- Dual Query Modes: Simple search syntax (e.g., level=error service=api) for quick looks, and full SQL for complex analysis when you need to dig deep.
- Lightweight & Focused: Single binary, easy to deploy. It doesn't do ingestion, letting you use best-of-breed tools for that.
- Team-centric: Built with multi-tenancy for teams and access controls for different log sources from the get-go.
I've got a demo running at https://demo.logchef.app if you'd like to try it out. It's still evolving, so feedback (especially from other ClickHouse users) would be super valuable!
So, Logchef has a concept of "Source" which represents a ClickHouse table. You give the DSN (essentially host/port/user/password for now) and connect. In prod scenarios, usually you only `GRANT SELECT ON db_name.table_name TO user_name;`
Once you add the source, you can "connect" the source to a team. Only the members of the team will be allowed to query this source. So you can have multiple teams and add users accordingly. A source can be added to multiple teams as well.
What I don’t like about Claude Code is why can’t they give command line flags for this stuff? It’s better documented and people don’t have to discover this the hard way.
Similarly, I do miss an —add command line flag to manual specify the context (files) during the session. Right now I pretty much end up copy pasting the relative paths from VSCode and supply to Claude. Aider has much better semantics for such stuff.
Personal blog I update every now and then. Mostly infrastructure, networking, and observability stuff.
reply