• 0 Posts
  • 10 Comments
Joined 6 months ago
cake
Cake day: March 26th, 2024

help-circle



  • Believe what you will. I’m not an authority on the topic, but as a researcher in an adjacent field I have a pretty good idea. I also self host Ollama and SearXNG (a metasearch engine, to be clear, not a first party search engine) so I have some anecdotal inclinations.

    Training even a teeny tiny LLM or ML model can run a typical gaming desktop at 100% for days. Sending a query to a pretrained model hardly even shows up on HTop unless it’s gigantic. Even the gigantic models only spike the CPU for a few seconds (until the query is complete). SearXNG, again anecdotally, spikes my PC about the same as Mistral in Ollama.

    I would encourage you to look at more explanations like the one below. I’m not just blowing smoke, and I’m not dismissing the very real problem of massive training costs (in money, energy, and water) that you’re pointing out.

    https://www.baeldung.com/cs/chatgpt-large-language-models-power-consumption


  • I don’t disagree, but it is useful to point out there are two truths in what you wrote.

    The energy use of one person running an already trained model on their own hardware is trivial.

    Even the energy use of many many people using already trained models (ChatGPT, etc) is still not the problem at hand (probably on the order of the energy usage from a typical search engine).

    The energy use in training these models (the appendage measuring contest between tech giants pretending they’re on the cusp of AGI) is where the cost really ramps up.


  • Is there a reason you’re not considering running this in a VM?

    I could see a case where you go for a native install on a virtual machine, attach a virtual disk to isolate your library from the rest of the filesystem, and then move that around (or just straight up mount that directory in the container) as needed.

    That way you can back up your library separately from your JF server implementation and go hog wild.


  • sunstoned@lemmus.orgtoSelfhosted@lemmy.world*Permanently Deleted*
    link
    fedilink
    English
    arrow-up
    1
    ·
    edit-2
    2 months ago

    Syntax-wise, it’s meant to be identical. I got on board when they were the only ones that enabled rootless (without admin privileges) mode. That’s no longer the case since rootless docker has been out for a while.

    I’m personally a fan of the red hat docs and how-to’s on podman over the mixed bag of tech bro medium articles I associate with docker.

    At the end of the day this is a bit of a Pokemon starter question. If your top priority is to get a reasonably common and straightforward job done just pick one and see where it takes you! :)


  • Syncthing is my answer though I appreciate it doesn’t get to the root of your question.

    There are local backups that include your system settings, text messages, contacts, call history and (optionally) apps. The one thing I want is the ability to pick a directory for the local backup so I can make it work with syncthing without jumping through hoops.

    It’s also compatible with Nextcloud and WebDAV if those are options for you.