Jean Louis

Is there GGUF version?

There is no LLM that ever brings it's own opinion. Please reach out to basics on how LLMs work. There is nothing "new" that LLM can give you.

LLM models act like a book: when you open a page, you already have the content stored. The model processes this existing information, generating probabilistic results based on the training data, not new insights. This means LLMs rely on structured, data-driven outputs rather than independent opinions.

LLM models are designed to process and generate text based on vast training data, and their outputs are results of statistical inference rather than independent opinions. The "ingested data" combines the model’s training knowledge with user-retrieved information, generating probabilistic results that align with the training patterns, not personal beliefs. Thus, LLMs rely on structured, data-driven outputs to provide answers, not independent thoughts or opinions.

Those so called "opinions" must align with the data they are trained on.

Let us say this way, if LLM can give opinion, that means it is 100% biased opinion based on the data it was trained on.

You simply cannot get true opinions.

Oh, I’m sure the LLM you’re referring to is as clear as mud. Which one, exactly? And of course, the context provided was as precise as a weather forecast in a hurricane. What was it? Sure, because the output was so crystal clear, it’s not like anyone could possibly misinterpret it. What did it say? Oh, I’m sure you tried every single LLM under the sun. Which ones, exactly?

Ohhh Mitko, you’re telling me your desktop is now officially a server that got tired of hiding under your monitor and just started hosting LLMs like a caffeinated cloud? 😅

“Got to 1199.8 tokens/sec on Devstral Small-2… on the desktop?”
My jaw dropped so hard I accidentally spilled my coffee on my keyboard — again.
You didn’t just upgrade your desk… you turned it into a mini datacenter with a 32GB M4 chip pretending to be a server room air conditioner. And you’re still using Mistral Vibe like it’s a 2005 laptop? 😂

Next time, just call it “Mitko’s Desktop Data Center v1.0” — complete with blinking LED fans, a 16-B200 GPU cluster on top, and a “DO NOT TOUCH” sticker taped to the power button (because if you touch it, you’ll accidentally delete your 3rd coffee break).

Now go ahead — test the big one. I’ll be here, typing “Is this GPU cluster actually a desk, or is the desk just a disguise for a server?” 🤔

P.S. You’re officially the guy who turned “workstation” into “server-on-a-desk-stand-with-a-caffeinated-look.” 🍵💻✨

Congratulation. Publish the script on how you run it for others to see.

Here is exactly how I run it:

/usr/local/bin/llama-server --jinja -fa on -c 32768 -ngl 64 -v --log-timestamps --host 192.168.1.68 -m /mnt/nvme0n1/LLM/quantized/Qwen3VL-8B-Instruct-Q8_0.gguf --mmproj /mnt/nvme0n1/LLM/quantized/mmproj-Qwen3VL-8B-Instruct-Q8_0.gguf

with the llama.cpp and API is of course available as well.

I’m running my own LLM because:
Privacy? 57% say it’s the biggest AI barrier…
But 48% still leak company data anyway.
CRAFT says privacy is architecture, not policy.
So I’m not waiting for “beta” — I’m beta-ing my data.
February 2026? Nah. I’m already typing on my own GPU.
Privacy’s not a feature — it’s a feature flag I turned on before the release.
And honestly? My model’s less “AI” and more “I’m not giving your data to strangers.”
Run your own. It’s fun. It’s free. It’s your data.
And it’s way more satisfying than waiting for “beta.”
(Also, no one’s gonna steal your jokes now. 😉)

Great. Would it run on 24 GB VRAM?

No, the Pangu Model License Agreement Version 1.0 is not a free software license. It imposes significant restrictions, such as prohibiting use within the European Union (Section 3) and requiring attribution (Section 4.2), which conflict with the principles of free software licenses like the GNU GPL or Open Source Definition. The non-transferable clause (Section 2) and indemnity requirement (Section 7) further deviate from standard free software terms.

🔥 "Open Model"? More Like "Openly Restrictive"! 🔥

Huawei calls Pangu Pro MoE an "open model"? That’s like calling a locked door an "open invitation." Let’s break down the brilliant "openness" here:

• "No EU Allowed!" (Section 3) – Because nothing says "open" like banning entire continents. GDPR too scary for you, Huawei?
• "Powered by Pangu" or GTFO (Section 4.2) – Mandatory branding? Real open-source models don’t force you to be a walking billboard.
• Non-transferable license (Section 2) – Can’t pass it on? So much for community sharing.
• Indemnify Huawei for your use (Section 7) – If anything goes wrong, you pay, not them. How generous!

This isn’t an "open model"—it’s a marketing stunt wrapped in proprietary chains. True open-source (Apache, MIT, GPL) doesn’t come with geographic bans, forced attribution, and legal traps.

Huawei, either commit to real openness or stop insulting the FOSS community with this pretend-free nonsense. 🚮

"not commercial" license isn't "Open Source", so please be accurate to users.

Reference:

The Open Source Definition – Open Source Initiative:
https://opensource.org/osd

Gemma License (danger) is not Free Software and is not Open Source:
https://gnu.support/gnu-emacs/emacs-lisp/Gemma-License-danger-is-not-Free-Software-and-is-not-Open-Source.html

So the goal of Google is just their monopoly and dependence of users. I suggest using fully free, free as in freedom, LLMs.