Skip Navigation
InitialsDiceBearhttps://github.com/dicebear/dicebearhttps://creativecommons.org/publicdomain/zero/1.0/„Initials” (https://github.com/dicebear/dicebear) by „DiceBear”, licensed under „CC0 1.0” (https://creativecommons.org/publicdomain/zero/1.0/)HE

A software developer and Linux nerd, living in Germany. I'm usually a chill dude but my online persona doesn't always reflect my true personality. Take what I say with a grain of salt, I usually try to be nice and give good advice, though.

I'm into Free Software, selfhosting, microcontrollers and electronics, freedom, privacy and the usual stuff. And a few select other random things, too.

Posts 5
Comments 1.1K
Large language models surpass human experts in predicting neuroscience results
  • I think this isn't really about predicting something. That's just a means to benchmark AI. You can either ask it questions to probe knowledge. Or test if it can look forward, reason and jump to some conclusions. In other words predict something. They tried how well it performed at that. Not because these predictions itself are useful. But because you can use them to measure the AI's capabilities at similar tasks.

  • Self hosting LLMs on a remote VPS
  • What's the difference regarding this task? You can rent it 24/7 as a crude webserver. Or run a Linux desktop inside. Pretty much everything you could do with other kinds of servers. I don't think the exact technology matters. It could be a VPS, virtualized with KVM, or a container. And for AI workloads, these containers have several advantages. Like you can spin them up within seconds. Scale them etc. I mean you're right. This isn't a bare-metal server that you're renting. But I think it aligns well with OP's requirements?!

  • Self hosting LLMs on a remote VPS
  • Well, there's both. I'm with runpod and they bill me for each second I run that cloud instance. I can have it running 24/7 or 30min on-demand or just 20 seconds if I want to generate just one reply/image. Behind the curtains, it's Docker containers. And one of the services is an API that you can hook into. Upon request, it'll start a container, do the compute and at your option either shut down immediately, meaning you'd have payed like 2ct for that single request. Or listen for more requests until an arbitrary timeout is reached. Other services offer similar things. Or a fixed price per ingested or generated token with some other (ready-made) services.

  • Self hosting LLMs on a remote VPS
  • That depends on the use-case. An hour of RTX 4090 compute is about $0.69 while the graphics card is like $1,600.00 plus computer plus electricity bill. I'd say you need to use it like 4000h+ to break even. I'm not doing that much gaming and AI stuff, so I'm better off renting some cloud GPU by the hour. Of course you can optimize that, buy an AMD card, use smaller AI models and pay for less VRAM. But there is a break even point for all of them which you need to pass.

  • Pointless AI server farms coming to an unregulated ocean near you!
  • I think they fly those technicians out by helicopter. There are some offshore wind farms. And as far as I know they work, and they're not particularly high maintenance. But you're right. Both maintenance and initial investment to build them are way higher than for wind on land. And seems even with all the wind on sea, they don't make up for that, resulting in a higher cost for the energy than what's possible with wind turbines on land. I did not know that.

    You're right. I kind of missed the international waters part. That's a completely ridiculous idea. And probably also requires some riduculous arrangements.

    I'd say judging by this calculation, we're looking at like 15MW of power for one of those platforms. The internet says solar gets you about 200W per square meter. Let's make up a factor of 3 if we have like 8h of sun and we need to store energy for the night. That'd be 225,000m² of solar panels. Or like 42 US football fields. If they're packed and you're at the equator. Or 5 wind turbines. Or they want some of those small nuclear reactors that currently get hyped by the big AI companies.

  • What are some illegal things that should be legal?
  • You're right. Looking at the list again, there are a lot of countries missing. Like Finland, which vaionko mentioned and several others from all kinds of continents. But I think what you said also applies to a lot of other democratic and free countries in the list. For example if I look at the list of Germany, where I'm from... That mainly lists books that include holocaust denial (which is a crime here, due to history) or other misinformation concerning that. And instructional books on how to build bombs or poison someone. So it's not like our courts are banning books without a proper reason, either. And I think it mainly concerns distributing and selling those books. Owning them is fine, with more or less the one exception of child pornography.

    And it's not the government's job to ban books here, either. These are some individual(?) court rulings.

    But with that said, my country isn't at the top of freedom of speech. I think we're cutting down on libel and defamation more than some other countries. And sometimes an author or publisher gets sued for publishing a book containing doxxing or lots of personal information abot celebrities/polititians without their consent. And then that's effectively banned from being distributed.

  • Pointless AI server farms coming to an unregulated ocean near you!
  • But in turn they generate less power, meaning they maybe have to pay extra for power. And run a cable to the platform. And they're talking about batteries, meaning they'd need to maintain that instead and that also requires experts. So yeah, idk. I'd say it's likely much more expensive that way, and they'd better train some engineers to maintain wind turbines. Those are build to last like 30 years. And wind power is supposedly cheaper anyways, with maintenance included in the calculation.

  • Have the patents for H.264 MPEG-4 AVC expired yet?
  • I mean ffmpeg, GStreamer etc also provide the encoders and decoders. That doesn't make it legal. I think they're all (including VLC) threatened by software patents. But you're right. There are differences between the EU and other jurisdictions.

  • Is it possible to have a "free speech" platform that simultaneously stops "hate speech"?
  • Well, if you allow everyone to say everything, the one yelling the loudest wins, and the more silent people don't get to speak freely. Also it's going to send hate, violence, doxxing, state secrets etc into the world. Harming other people and limiting their freedom. Or you limit free spech. So either way, there is no such thing as free speech. It contradicts itself.

  • Has Google's Tensor project failed?
  • Do they even offer something of value? I mean for that they'd need to have something like a good and open SDK to appeal to developers, who then use the specific advantages of those chips. Which in turn does something for the consumer to chose their chips over their competitor's ones. And as far as I'm concerned, the camera app is the only thing using the NPU and GPU parts of the chip. So I really don't care if it's a Tensor or a Snapdragon. As long as it's fast enough for what I do and efficient with the battery power.

  • How can I add a simple requirement "do not train Al on the source code of the program" to AGPLv3 or GPLv3 and thereby create a new license?
  • Indeed. I hope so. And we desperately need some clear regulations. Even the big AI companies struggle with the lack of clear rules. I can see how we need to go through quite some legal battles to settle some questions arising with the new technology. And that's currently taking place. But it extends past that. Currently, companies are retreating from the European market. Due to a completely unmanageable situation. I've seen local language models (starting with Llama 3.2) being banned / not licensed within the EU. And that's going to lead to all kinds of complications. Just because the EU can't get some proper regulations out, and do it in time. That'll leave technological progress behind in the EU, mess with companies. In effect also take away my freedom to run language model on my own hardware...

    I hope they get that straight. And there is some demand... So maybe it's happening sooner than later. But these are very difficult questions to answer. About AI safety, copyright, effect and impact on society and freedom... And I think a lot of these questions are difficult to tackle with licensing anyways. We definitely need laws governing if AI training is fair use. Or if generating a voice that sounds 70% like David Attenborough is alright to do.

  • How can I add a simple requirement "do not train Al on the source code of the program" to AGPLv3 or GPLv3 and thereby create a new license?
  • I mean I can also read your code, take inspiration from that and use it to write some proprietary software. I think that's at least somewhat similar to what happens with AI. AI doesn't reproduce the code verbatim, but instead learns from it and as far as I know nobody found it repeating large chunks of one specific software. So I'd say it's like me reading copyrighted computer science books, learning programming that way and nowadays using my skill to code Free Software (or whatever).

    I know this position is disputed. But I think there is some truth to it. At the same time it's close to the tech companies' rationale. It's morally wrong that they get to profit from other peoples' labor. And they're definitely exploiting the situation that law and licences come from a time, where AI wasn't an issue... But I'm really split on the topic. Ultimately we'd need some consensus on how to handle this. And some laws and regulation. And we don't have that yet.

    And I think it's also similar to other companies profiting off of FLOSS projects. Like with Redis, MongoDB(?) and all the projects that shifted from open-source to source-available due to Amazon et al just taking things and making profit by selling it as a cloud service without ever contributing back. Is just a sad situation. And ultimately it harms me and everyone. Because I'm subject to the same license. And now I can't use, modify and share some software anymore. These non-commercial clauses are difficult, too. Even if I just run a small Fediverse instance and collect donations, that could be construed as commercial. Or trying to make a living off of Free Software. And I think all of this drama is an even bigger problem than AI being trained on other people's code. And it all cuts down on freedom. I mean for a legitimate reason... And I get it... Still the freedom gets lost.

  • How can I add a simple requirement "do not train Al on the source code of the program" to AGPLv3 or GPLv3 and thereby create a new license?
  • Don't do it if you like the (A)GPL or free software licenses. It'll probably void the license. At least adding clauses immediately makes it incompatible with other open source software and take away user freedom. It's generally not recommended to do this. (Same applies to the commons clause and other additions, they usually tend to make our life as the free software community worse.)

    And it would only do harm, without any benefit. The companies claim training AI is fair use. And they'll continue doing it anyways. At the same time every hobby programmer who forks your project on Github and wants to contribute will be in breach of your license, as Github is known to feed that into Copilot...

    I'd recommend taking a step back and re-think this. Why are you giving away your project in the first place? Do you really care how people use it? If yes, your interests aren't completely aligned with the idea of Free Software, which grants the user 4 essential freedoms. Including using your project for arbitrary purposes. As the author, it's your decision. But the (A)GPL isn't really the correct license/vision if you don't fully agree with the premise.

  • Does having someone/something to lose make you weaker or stronger?
  • Both. There are situations where either of this applies. Or both. People regularly fight over things, land or other people. To their own benefit and it makes them stronger, or more determined. At the same time the same thing can be exploited by other people. Or even get lost due to circumstances, breaking someones spirit. It's all of this.

    And you better have something to lose, or there isn't much of value in your life.

  • media.fsfe.org SFSCON24 - Alexander Sander - NGI: No more EU funding for Free Software?!

    During the summer the European Commission made the decision to stop funding Free Software projects within the Next Generation Internet initiative (NGI). This decision results in a loss of €27 milli...

    SFSCON24 - Alexander Sander - NGI: No more EU funding for Free Software?!

    > During the summer the European Commission made the decision to stop funding Free Software projects within the Next Generation Internet initiative (NGI). This decision results in a loss of €27 million for software freedom. Since 2018, the European Commission has supported the Free Software ecosystem through NGI, that provided funding and technical assistance to Free Software projects. This decision unfortunately exposes a larger issue: that software freedom in the EU needs more stable, long-term financial support. The ease with which this funding was excluded underlines this need.

    > CC BY-SA 4.0 - SFSCON 2024

    Cross-posted from the FSFE Peertube Channel

    3

    Is Arli AI a legit cloud LLM inference service? Any user experience?

    I just found https://www.arliai.com/ who offer LLM inference for quite cheap. Without rate-limits and unlimited token generation. No-logging policy and they have an OpenAI compatible API.

    I've been using runpod.io previously but that's a whole different service as they sell compute and the customers have to build their own Docker images and run them in their cloud, by the hour/second.

    Should I switch to ArliAI? Does anyone have some experience with them? Or can recommend another nice inference service? I still refuse to pay $1.000 for a GPU and then also pay for electricity when I can use some $5/month cloud service and it'd last me 16 years before I reach the price of buying a decent GPU...

    Edit: Saw their $5 tier only includes models up to 12B parameters, so I'm not sure anymore. For larger models I'd need to pay close to what other inference services cost.

    Edit2: I discarded the idea. 7B parameter models and one 12B one is a bit small to pay for. I can do that at home thanks to llama.cpp

    4

    How to make the Threadiverse a nice place and effectively make it grow

    tl;dr: Be excellent to each other, do something constructive here?

    I'm not sure anymore where the Threadiverse is headed. (The Threadiverse being this threaded part of the Fediverse, i.e. Lemmy, MBin, PieFed, ...) In my time here, I've met a lot of nice people and had meaningful conversations and learned lots of things. At the same time, it's always been a mixed bag. We've always had quite some argumentative people here, trolls, ... I've seen people hate on and yell at each other, and do all kinds of destructive things. My issue with that is: Negative behavior is disproportionately affecting the atmosphere. And I'd argue we have nowhere enough nice behavior to even that out.

    I don't see Lemmy grow for quite some time now. Seems it's now leveling off at a bit less that 50k monthly active users. And I don't see how that'd change. I'm missing some clear vision/idea of where we want to be headed. And I miss an atmosphere that makes people want to join or stay here, of all of the places on the internet. The saying is: "If you don't go forwards you go backwards". I'm not sure if this applies... At least we're not shrinking anymore.

    And I'm always unsure if the tone and atmosphere here changes subtly and gradually. I've always disagreed with a few dynamics here. But lately it feels like we're on the decline, at least to me. I occasionally keep an eye on the votes on my comments. And seems I'm getting fewer of them. Sometimes I reply to a post and not a single person interacts. Even OP seems to have abandoned their post moments after writing it. And also for nuanced and longer replies, I regularly don't get more than one or two upvotes. I think that used to be a bit better at some point. And I see the same thing happening with other peoples' comments. So it's not just me writing low-quality comments. What does work is stating simple truths. I regularly get some incoming votes with those. But my vision of this place isn't spreading simple truths, but have proper and meaningful discussions, learn things and new perspectives or just mingle with people or talk. But judging by the votes I observe, that isn't appreciated by the community here.

    Another pet peeve of mine is the link aggregator aspect of Lemmy. I'd say at least 80% of Lemmy is about dumping some political (or tech) news articles. Lots of them don't generate any engagement. Lots of them are really low-effort. OP just dumps something somewhere, no body text added, no info about what's interesting about it. And people don't even read those articles. They just read the title and react (emotionally) to that. In the end probably neither OP nor the audience read the article and it's just littering the place. Burying and diminishing other, meaningful content. (With that said: There are also nice (news) discussions going on at the same time. And Lemmy is meant to be a link aggregator. It's just that my perception is: it's skewed towards low quality, low engagement and random noise.)

    A few people here also don't really like political debate. And there's no escape from it here on Lemmy since so much revolves around that. And nowadays politics is about strong opinions, emotions and emotional reactions. And often limited to that. The dynamics of Lemmy reinforce the negative aspect of that, because the time when you're most incentivized to reply or react is, when it triggers some strong emotion in you, for example you strongly disagree with a comment and that makes you want to counter it and write your own opinion underneath. If you agree, you don't feel a strong emotion and you don't reply. And the majority of users seems to also forget to upvote in that case, as I lined out earlier. And we also don't write nuanced answers, dissect complex things and examine it from all angles. That's just effort and it's not as rewarding for the brain to do that as it is pointing out that someone is wrong. So it just fosters an atmosphere of being argumentative.

    Prospect ----------

    I think we have several ways of steering the community:

    1. Technology: Features in the software, design choices that foster good behavior.
    2. Moderation: Give toxic people the boot, or delete content that drags down the place. Following: What remains is nice people and not adverse content.
    3. The community

    I'd say 1 and 2 go without saying. (Not that everything is perfect with those...) But it really boils down to 3: The community. This is a fairly participatory place. We are the ones shaping the tone and atmosphere. And it's our place. It's kind of our obligation to care for it if we want to see it go somewhere. Isn't it?

    So what's your vision of this place? Do you have some idea on where you'd like it to go? Practical ideas on how to achieve it? Do you even agree with my perception of the dynamics here, and the implications and conclusions I came up with?

    82

    Just testing...

    I'm just testing if I broke something. But that doesn't seem to be the case, since it got federated and you can read this. You can stop reading now.

    And since you didn't... How do you like Piefed? Do you also use Lemmy along with it because of some shortcomings? Do you miss a phone app? Do you operate your own instance (or plan to) or are you comfortable using piefed.social? What are your plans for it, just using it for yourself? Are you able to code in Python? And do you think Python and Flask are good choices? [Edit: Edited to test editing.]

    0