>>6258 good luck!

lots of /lmg/ refugees in https://meta.4chan.gay/tech/67288

>>6266 I'm curious to see where everyone will consolidate

>>6258 omg it migu

>>6270 I want 4chin back...

>>6273 It'll be back eventually and probably worse than ever

>>6273 4fag mods and jannies are troons, we are the mods and jannies here

>>6266 >here are your neighbors, bro

https://huggingface.co/microsoft/bitnet-b1.58-2B-4T https://github.com/microsoft/BitNet In case anyone missed it in the chaos, microsoft actually trained a bitnet model. It's a 1.58b so more of a retard you can carry around in your pocket than anything useful but I suppose it's proof that bitnet isn't a completely abandoned concept.

>>6286 anons tested it out already, okay for a 2b model https://meta.4chan.gay/tech/67288#p76975

>>6287 >Serbia >Solarized theme hi petra

>>6258 Is that new 47b Nemotron model roleplayable like the recent 49b one, or is for researchy stuff?

>only options are here, dead, or the literal cunny chan what the fuck

>>6293 What is the cunny chan name?

>>6293 at least here we have post ids, but yeah all of the options suck

hello where did Hentai Diffusion go?

>pedophiles all flock to a literal pizza altchan hmmm

>>6349 https://meta.4chan.gay/tech/67288 use fennec f-droid or any other firefox based browser on mobile if you have issues posting

>>6428 GO AWAY POO POO NIGGER MORAL FAG FAGGOT THIS IS OUR BOARD NOT YOUR FUCK OFF TO INDIA OR TURKMENISTAN OR WHEREVER YOUR SHITTY UNWIPED BUM WAFTED IN FROM, THIS IS NOT YOUR SHITTING STREET, THIS IS OUR SHITTING STREET, NOT PUBLIC, NOT FOR YOU

>>6258 I am home again

>>6412 /trash/ got their sdg back, but I haven't found something like Hentai Diffusion yet In the meantime your best bet might be civitai?

>>6287 I got it running now as well. Hope they will continue experimenting with Bitnet

>>6273 No way. Seeing the solo janny in /h/ getting doxxed was funny.

>>6568 >4chan acquired by Y Combinator Fate worse than death.

>>6293 /g/ was always the technololigy board, fag.

>>6266 Nice try. I'm not going to any site with ".gay" at the end of the URL.

uhh.. guys? anyone alive?

>>6266 Your shit is down

>>6647 yeah well, if you checked the archive you'd know that ALL /lmg/ refugee locations are regularly posted there

https://meta.4chan.gay/tech/67288 WE'RE BACK! MASSIVE HAPPENINGS HAPPENING

we got 2 /lmg/ now? I'm liking this better.

its OVER!

Was just up a second ago.

ok since 4chan gay is being gay lets talk local models whats up anons

4chan.gay is gay altchans suck

>>6669 4chan.gay's /lmg/ was better than this ghost town. too bad the 4chan.gay admin is a dipshit who tests in prod

4chan gay is cool, but whoever is managing it is some ADHD zoomed retard. I guess 4chan is as great as it is because the management never is present...

its back again https://meta.4chan.gay/tech/67288 ...

4chan itself was gay. no vpns, countdown timers. these alt-chans are at least anonymous. I would rather have one take off.

>>6706 None of them work without javascript 4chan.gay hosts CP while being behind cloudflare. It's the glowiest honeypot to ever glow

some more news

>>6707 >>6712 >Reporter's Name: Hiroyuki Shouldn't he be more concerned about bringing 4chan back up instead of attacking the competition?

person who reported inside https://unknown.spam/aicg_mail_list

>>6707 >>6710 We're posting about models not CP. I don't give a fuck, may the strongest chan win. Would you rather reddit or discord?

>>6717 matrix

>>6718 I tried that. It was psychotic leftists.

>>6720 theres a few based homeservers, although any platform similar to discord will eventually lead to 'cordfaggotry so i'd rather we keep it on literally any chan

There's lainchan too you know, the place seems comfy

>>6724 extremely cancerous trannie jannies

>>6725 Considering it's you, I bet they banned you for shitting the place up and you're butthurt All the more reason we should consider lainchan

>>6727 *stands in your way* your move?

>>6725 It's no better than gay-chan, the mod is watching as we speak

>>6725 >>6727 The fact that lainchan doesn't have any threads for AI suggest they are not very interested in it (or anything too new actually). Also, the ai generals would be far too fast for them. Here is better for now. The gay 4chan is not working for me.

Someone please bake /ldg/ in this board please

>>6772 https://meta.4chan.gay/tech/67288?last=100#bottom works like this if you're a ramlet or something

>>6717 >dude just ignore the Democrat activism next door >If you don't like it then you must want to go to reddit or discord instead!

>>6837 >cunny.. is LE BAD

https://seed-tars.com/1.5/ https://huggingface.co/ByteDance-Seed/UI-TARS-1.5-7B VLM from bytedance, focused on computer use. Might be interesting. A lot of other computer use systems have basically been just bolting one of the obese models onto a browser use system. This seems relatively more polished and better for interactions, but I have doubts about its ability to handle more complex tasks.

EXL3 with cache quantization when?

I want to chat with a chinese LLM and see if its views about china differ from western ones. Which one should I check first? I can run up to 32B. GLM? qwen? qwq?

>>6921 see https://meta.4chan.gay/tech/67288?last=100#p106370

>>6921 Yeah if you want chinese models try qwen's stuff, GLM, deepseek's if you can get it running. btw If you're just doing quick evaluations then you might have a better time just trying them out on openrouter rather than downloading every single one.

>>6921 qwq is the quintessential local Chinese model atm.

>>6854 When turboderp gets time off his dayjob and finishes railing his anime girls.

Bros! You're back!

>>6976 GLM still has an open PR in llama.cpp for some problem, I will wait. I see that qwen has official gguf quants in hf. I will test 2.5 and qwq. I prefer to use 100% local, especially if I want to test the "limits" of a model.

>8chan has miku theme We're so back it's unreal.

>.moe is literally dead >4chan gay is figuratively dead >desuarchive was never actually alive It's unironically over

>>7087 Shit... that could be a while.

If 4chan doesn't come back, the canonical /lmg/ is going to be wherever the thread recap bot operator and/or CUDA dev show up. This place looks ok so far, so maybe there's hope!

>>7209 Recap Anon is here and in 4chan gay, so it's actually up to whichever place has more anons. I wonder about CUDA anon... I will try to send him a email.

>>7200 It's not over, fren. The first reaction of most people was to wait it out, expecting 4chan to come back online in short order. With every day that passes, more and more of those people are starting to look for alternatives. They'll find us.

I've come here to complain that even though jetbrains recently added support for local models in their ai shit it's still worse than zed's.

>>7235 >jetbrains >zed This feels aliencoded.

>>7235 local llm aren't for real work

>>7249 qwhen? 3 will make local LLMs viable for real work.

>>7235 >>7237 >>7249 Petra, stop doing this

Am I retarded? Why does this guy recommend 512x512 for wan when it's not in the recommended resolutions? https://comfyanonymous.github.io/ComfyUI_examples/wan/

>>7275 because he's a fucking retard

>>7249 They cover a good chunk of it if you care enough about the ideology behind running local. The simple boilerplate, small changes, relatively simple bugfixes, can be handled just as well by current 70Bs as they can by e.g. Gemini Flash. (for me, deepcogito 70B and before that, Athene) I just really don't like the idea of individuals completely losing the ability to do their own computer stuff on their own hardware. So yeah I won't be so ridiculous as to never use the cloud stuff, when it really calls for it, but when I'm using local models it makes me feel like "you will own nothing and be happy" hasn't progressed quite so far.

>>7324 deepseek v3/r1 is also local.

MAIDS save local models https://huggingface.co/microsoft/MAI-DS-R1

>>7350 >MAI-DS-R1 is a DeepSeek-R1 reasoning model that has been post-trained by the Microsoft AI team to improve its responsiveness on blocked topics and its risk profile >MAI-DS-R1 has successfully unblocked the majority of previously blocked queries from the original R1 model Microsoft uncensoring models? I somehow doubt it. If Microshit got their claws on it, then they may have unblocked it's ability to tell you about Tiananmen Square, but at the cost of losing the ability to tell you what a woman is.

>>7350 no

>>7353 I care less about that aspect than the slight hope that the finetune lessened R1's chaotic adhd tendencies as a side effect. It's cope but tunes by big corpos like this are likely the only real ones we're going to see for Deepseek considering the size of these models. I just wish there were quants for it.

>>7353 It's a double edged sword. They made it so you can ask about Tiananmen on the model but in return, they trained it on the same safety mix as Tulu so it went full safety from a Chinese point of view to a Western point of view. It is marginally better for real tasks like code generation due to the better data that Microsoft added but I would hardly say that was worth it. But Microsoft used those compute resources, not us and it's for enterprises so makes sense.

>discount /lmg/ hours >and discussing a fucking fine-tune that nobody should give a shit about What a fucking retarded discussion. Put this general out of its misery.

>>7358 >having a mental breakdown over people discussing one of the few finetunes for one of the best local models we have Is being poor that hard on you?

>>7363 >one of the few finetunes It's the exact same thing that Perplexity already did, the only thing all those companies care about is swapping Chinese propaganda with an American one. And then there will be /r/LocalLLaMA-level retards that will shill the model like if it became "uncensored". It's all those American companies the ones adding censorship we care about in in the first place. Fuck you for posting it here.

>>7364 Let people chose the propaganda they want dude.

>>7379 Not gonna get excited for western cucked models. Even if they benchmaxx a little higher.

Does the sillytavern image generation function not work with REFORGE? does it have to be the old A11111 SD1.5 UI? I upgraded to reforge ages ago and it cannot seem to find the connection to my reforge when I'm running it

>>7465 I've had good success with ComfyUI that's what everyone seems to be using for everything imagegen these days...

>>7465 It worked on the old re-forge made by pancho. I dunno about the new one. After he stopped updating, I moved to comfy.

>>7467 >>7468 I have never used comfyui for anything. How do you launch it so sillytavern picks it up? or better yet is there a guide for sillytavern image genning with comfyui? I just want to be able to have images be genned based on the situation mid-RP

>>7469 You start it with the API active, make a workflow and then put that WF with stuff like prompt replaced via placeholders inside silly. Not as plug and play like A1111 was but lets you do a whole lot more.

Any news about Qwen3? I missed the last couple of days because of the whole 4chan thing.

Well according to the system message on 4gay they're getting shut down. So I guess this is the official /lmg/ now.

>>7489 qwen3 miku oo ee oo

RIP. Perception-LM-8B ooms on a 3090. Useless model.

https://8chan.se/bot/ Our own board.

>>7593 We made a measly 100 posts in 4 days. Why would you want to splinter off now?

>>7593 no thanks

>>7364 >>7364 There's a good reason for them to do this finetune that has nothing to do with us using it as R1 was essentially mostly uncucked for most purposes anyone here would care about. Retarded politicians in Washington want to ban open weights model R1 because it was made in China and keep grasping at straws for some reason to ban it (not that there's many), but since this is MIT licensed, Microsoft is probably doing some legal trolling where they would finetune it and show some use and thus could defend it in court if the boomers do end up attempting to ban it. Obviously such a law would be unenforceable and they would be shooting themselves in the foot, and code is speech and all that, but Microsoft having their own variant would probably count as a good start for a defense.

>>7668 Also, isn't it R1 the strongest model you can run locally right now? This could be useful for companies with pockets deep enough to run R1, but in need of a model aligned to western sensibilities.

>>7679 yes that is the only usecase

>>7679 There was already one such finetune that came out weeks after R1 came out. Mostly though R1 isn't even that heavy on the refusals on the one thing that they tuned it against (CCP stuff), a simple prefill will avoid most issues as usual. And yes, it's close to the best open weights model currently.

>>7689 close?

>>7695 For example a reasoning finetune of 405B can reach similar performance to R1, Nvidia did one recently. It also depends on your usecase, sometimes you may be fine with a dumber model that uses less VRAM. Also, the first DS3 on which R1 was based on had serious repetition issues (somewhat solved in 3.1), which some smaller models (such as mistral large) lacked.

>>7703 ugh fine, but its so safety cucked . . .

>>7706 I'd just use R1, but I guess it's not uncommon for models to need some finetune after to remove "safety". Base models tend to be uncucked, but if the dataset is too filtered, the output can be too plain/boring, so ultimately you still need a finetune on top of it.

>>7707 >>7703 why would anyone want to use a 253B dense model over a 37B/671B MoE? if both have same-ish performance

>>7708 idk, I haven't played with nvidia's tune, but maybe there's some reason? It's like asking why would someone prefer claude opus or sonnet 3.7 over R1 or whatever, might depend on taste and how it performs in specific tasks. Currently R1 could be better at tool use, it's not like they don't have things to improve. I wonder if R2 will handle those well.

>>7708 No consumers at least. Can't run 250B on RAM without killing token generation speed.

Just want a quick update since I haven't been keeping up. Is Nemo still unbeaten by a model same parameter count or less? I'm guessing yes because it's a safe bet at this point, but figured I'd ask

dead thread, dead website, dead hobby

happy Easter

Just got myself a 3090, what the best model I can run for peak kino AI lewd roleplays?

>>7812 post rest of your specs, also https://meta.4chan.gay/tech/67288?last=100#bottom is more active

>>7812 cydonia

>>7812 MS-Magpantheonsel-lark-v4x1.6.2RP-Cydonia-vXXX-22B-8.i1-IQ4_XS.gguf

>comfy thread >growing website >developing hobby

>>7747 >https://huggingface.co/OnomaAIResearch/Illustrious-XL-v2.0 >Illustrious XL 1.0-2.0 series aims to stabilize native generation at 1536 resolution while significantly improving natural language understanding capabilities. Not really that interesting, I think it is hitting against the limits of what SDXL can do without Vpred. I expect a lot of models to probably rebase on this since we will probably never get local 3.0/3.5 Vpred from Angel and how funding has essentially almost stopped. >https://huggingface.co/OnomaAIResearch/Illustrious-Lumina-v0.03 >This model is based on Alpha-VLLM/Lumina-Image-2.0 , which is nice small DiT model with minimal guaranteed functionality! Please refer to https://github.com/Alpha-VLLM/Lumina-Image-2.0 for official repository. This is interesting but I suspect he tried to train it before their technical report was out. Lumina was trained on extremely details and long captions for tags and boomer prompting and they even built their own tool for that. I suspect the training wasn't as effective as it should've been because of that, and as the model says, it can recognize characters now but it is still severely undertrained to the extent where it doesn't even equal the training done on Illustrious v0.1

What's with the fake 404 on 4gay?

How about model for sci-fi novel slop?

>>7940 4chan got pwnd by sharty

>>7946 i mean 4chan.gay

>>7814 Will those niggers just come here instead I'm not going to a pizzachan

>>7958 they'll pick literally anywhere else but here. is it because of muh ids?

I get this red text each time I launch silly. What exactly is this and how do I fix it, idk where exactly it wants me to click for this. I've ignored it so far

>>7960 Choose Text Completion on the 2nd dropdown list under API text.

>>7959 It actually is ids, lmg has a history of randomly being spammed (by soijack party users no less) so obviously they won't post here

>>7358 >Implying that 50% of /lmg/ discussion wasn't always about trying out whatever new meme finetune

>>7962 I'll give it a try next time ty

>>7965 i love ids

>>7975 Based. Easy to get around though. I post through a vpn and get a new ID every time without changing anything. Not intentional, I like IDs.

>riverwind Is this a trolling model? I keep getting shilled by products.

>>7999 kek

>>8001 >001 AAAAAAAAAACCCCCCCCKKKK

>>7999 yes its a troll model, unironically great at what its made to do

>>7999 pretty sure it was an april fools day project that wasnt ready in time

>>7959 Probably, the guy who makes most of the posts there replied to himself here twice >>7237 >>7264 >>7275 >>7276

>>8021 What the fuck lmao what a weird cunt. if 4chan ever comes back IDs need to be on every board to out freakshows like this

>>8021 Why the fuck are You giving him (You)s

>>8021 What causes one to behave this way?

>>8028 you's are not currency dont be a faggot, this person deserves to be pointed out and shamed

>>8029 Mental illness.

https://github.com/JohannesGaessler/elo_hellm >Elo HeLLM is a project for establishing a ranking based on Elo ratings between large language models. The context is that I'm working on training code for llama.cpp. llama.cpp has methods for estimating the quality loss from quantization but it lacks methods for estimating the quality of a model in absolute terms or for making comparisons between different models. I intend to co-develop this project with the llama.cpp training code for quality control. The approach is to merge an arbitrary number of quality metrics into a single Elo rating for a model using statistical methods. One category of such quality metrics are simply the results of language model benchmarks such as MMLU. Results from competitive games such as Chess can also be used (not yet implemented).

Hey wait wtf. I just noticed that my post here >>7237 has the same ID as a bunch of other posts in the thread that aren't mine. I'm serious. Also, I don't see "(You)" in the replies. I'm getting spooked what the hell.

>>8036 Why is my id different ahhhhhhh.

>>8036 fake until proven gay

>>8038 >petrified petra is a gorgon

>>8038 >>8039 But seriously though this is creepy. Are the mods messing with me? Did I get hacked? How am I even supposed to get proof in this situation?

>>8041 Why do you care? Even if you are telling the truth you are anonymous and have no identity worth protecting.

>>8041 Your IP could have changed and some other guy has gotten your exact previous one. Which is probably less likely than winning the lottery.

serial expetriments lain

>>8041 In all probability, someone is just using the same VPN.

>>8046 this is true. watch me change me id by changing my vpn

>>8047 Sex with AI.

>>8048 as shrimple as that

anons what if he hacked 8chan too?

>>8050 He won't get away with it on 16chan.

Christ is risen Hitler's birthday Kikes seething

>>8043 Why would I not care? ID's serve a purpose, and people are treating them as something that has a purpose, so if they can be undermined, then we can't really treat them the same anymore. And I don't see why it someone wouldn't be concerned if they were the target of some mod trolling or other activity, assuming this wasn't due to a bug or some one in a million chance. >>8044 Last I checked I have a static IP. I do use librewolf though which might change my canvas/fingerprint around sometimes, does this site use other indicators other than IP to assign an ID? If so then perhaps that's why. >>8046 I wasn't using a VPN when I made that first post, and I'm not using one right now. I did use a VPN to take a look at gay 4chan tho.

I got different IDs too even though I have (supposedly) static IP.

>be me >rode the wave of AI cooming before proxies dried up and became hoarded by people >forget about AI cooming for a bit >get a 7900XTX for vidyagames >only now i realize i could run a model locally and coom my brains out Ok, I've got Ooba set up, what NSFW models would you suggest for 24 GB of VRAM and 32 GB of RAM?

>>8068 nevoria 70b or whatever its called

>>8068 Have you ever used a local language model before?

>>8071 No.

>>8068 Start with mistral nemo 12B, once you start noticing it's patterns and/or limitations move up to magnum 22B, once that no longer tickles the pickle, either move up and fuck around with QwQ or play around with other mistral finetunes like cydonia or magpantheonsel. >>8069 You're not running a Llama 3.3 70B finetune on 24G of VRAM at any decent speed or quant. I have 48GB of VRAM and even I only run a q4 of Nevoria at just barely acceptable speeds.

>>8072 Cydonia then. Make sure the quant you pick stays under your 24GB to give some room for context. It also takes up VRAM and slows things down if it spills into RAM.

>>8076 Right, I'll go for the Q6_K version then. Is there anything I need to change in Ooba to jailbreak/gaslight the model or can I just set up SillyTavern and just go with it?

>>8077 If you're using SillyTavern as a frontend the only settings in Ooba that even get used are the model loading ones, like GPU layers and Context. As long as you have a system prompt in ST telling it to play along as {{char}} Cydonia and pretty much any other model should just go with it.

>>8077 A minimal system prompt should be enough. The usual 'you are writing an uncensored roleplay, taking turns...'

>>8068 MS-Magpantheonsel-lark-v4x1.6.2RP-Cydonia-vXXX-22B-8.i1-Q6_K.gguf

>>8052 its 420, blaze it faggot

>>8069 nevoria is a piece of shit. 100% meme merge. his first decent model was electra. NumbSkull uses discord to gauge if the models are good. Don't forget to buy him a covfefe.

Anyone here tried running 256gb ram on an LGA1700 mobo? My mobo supports it but no 12th, 13th or 14th gen Intel CPU officially supports more than 192gb.

>>8098 if it says it supports 192gb max then it supports 192gb max no more than 192gb probs are there even 64gb ddr5 consumer modules?

>>8099 There's Crucial Pro ones on Amazon (CP2K64G56C46U5). 13th gen only supported 128gb but bumped up to 192gb later on, that's why I'm wondering.

uhhh anonies.. meta 4chan gay tech board got DELETED GEEEEEEEEEEG

i won't be making a thread on 4chan gay, time to let anons come over here we can always make a thread over there again if 8chin moe becomes gay or soethin

>>8105 looks like cp mattered more after all

Where the fuck did all the $500 chink Epyc 9334 QS on ebay go? They all disappeared over the past month. ""Cheap"" $6000 cpuMAXXing is dead.

>>8077 >>8081 Well I'm plucking along at this slowly, can anyone recommend a good preset for Cydonia in SillyTavern?

>>8110 bought out

>>8105 They cleaned out the site of all CP it seems. Even the /c/ board is gone and now the admin is telling people to go back to their holes. Hilarious.

>>8119 It said right in the name that they were gay.

>>8116 for the ms mag mell shitmix https://files.catbox.moe/f6htfa.json

>>8119 It is quite amusing, but it doesn't seem genuine. EPI threads are still there.

for the lonely anons out there https://huggingface.co/bartowski/v2ray_GPT4chan-24B-GGUF

>>8119 Nah he's retarded, he didn't have to delete lmg and other perfectly fine threads. That nigger is a fed and his website is a honeypot and they just get rid of the honey

dont worry guys i have a recent enough archive on my encrypted ssd i will post it if you anons want it.. soon

>>8123 >but it doesn't seem genuine. It's not. It's just a reaction to cloudflare and the host getting on the admin's tail. Truth is, the people in charge are furry/zoophille/pedophiles (all three at once yah) who crave attention. The admin is a known avatarfag on 4chan for example. It's a clown show. Anyhow. Local models. >>8125 He mass deleted all threads that were older than a day or something like that because there was a lot of shit to be found on the website if cloudflare or the host provider were to go poking. Yes, he could have implemented a smarter approach, but they are also somewhat technologically inept.

aww it's gone. it was comfy but our neighbors were a little odd.

>>8110 I reckon everyone started to have the same idea, just like with used 3090s Anyway, so how we liking 8chan lads? I think the IDs are pretty damn sweet and we actually have a REAL /ai(s)/ board

>>8138 Everyone will go back to 4chin once it's back online, but it'll do for now.

>>8140 >>8141 >this is who accuses you of being petra

>>8142 just report and ignore

>>8140 blacked miku.... petroons... now this really feels like home.......

how to report bruh

>>8146 lurk moar

Like I was saying, IDs are great. Not infallible, but great

kek

>>8153 tbh, some fags on e6ai are doing some pretty good ai slop anims, but most of them are cloud based so, not lmg.

>filters have options for name and tripcode but not post id useless

>>8157 Easy to make an userscript for that, at least.

>>8152 They really are

>>8163 You're visible now retarded nigger

>>8159 feels just like the old lmg

>wake up >gay thread nuked >admin proved himself to be an absolute bozo >cunny replaced with fu**y shit as a cherry on top welp. 8chan it is then.

I hope that /lmg/ will settle here or at least not on 4chan.gay. I can't even post from a hardened browser on it, this shit is obviously doing some heavy fingerprinting.

>>8176 Either here or erischan would be good. I like it here for the IDs.

>>8179 IDs are gay, a important part of the chan experience is being able to samefag tbhdesu, I feel very limited.

>>8180 >samefaging kys

>>8180 Yeah, that's exactly why IDs are good.

>>8180 You can still do that. Comes with the privilege of being made fun of in a screenshot.

>takes down all of 4chin to scatter /lmg/ >tries to get anons on /ghost/ to come to 8chan where he is a mod >anons go to gay chan instead >report gay by including links to his posts to try and get the host to shut it down >doesn't work, but causes a freakout so admins do a purge >big thread gone >anons move here complete blacked miku spammer victory

>>8180 Just change your IP bro

>>8187 I really hope that headcanon isn't real, even discordniggers are less cancerous than that

test

>>8190 lmao

>>8187 >>8189 it's obvious to anyone that it's the opposite.

>>8190 Spoopy.

So what are you doing now /lmg/entlemen?

>>8180 Do it anyway, who cares if anons call you a fag for it. Or maybe you are one?

>>8195 Using Gemini to format and consolidate a fuckton of data. I can't wait until we have models good enough, software good enough, and hardware cheap enough to do that kind of thing locally. Might take a year or two, but we'll get there eventually.

>>8196 I may or may not be, and that's another reason I dislike IDs. I don't want my fagness status to be tied to my reply history. If I wanted to be bullied for being myself I would be on reddit.

>>8197 You tried structured outputs?

>>8124 I'm malding trying to use this shit. What template does it use?

>>8198 Retards ruin things for everyone. IDs are still better than full blown accounts at least.

>>8201 If only we had a file format where you could embed the prompt template so backends could know how to format text for the model. Maybe we could use jinja since AI fags love python so much.

>>8198 Working as intended. Don't make shit posts and IDs aren't an issue. If you fuck up take a time out and wait for the next thread.

>>8176 >his shit is obviously doing some heavy fingerprinting. Prob from constantly monitoring your message typing kek

>>8200 You mean like JSON schema or BNF grammar? I'm dealing with plain text output, not JSON, XML, or the like. Not that I couldn't use BNF for example enforce header sub-header paragraph, or something like that, but that's besides the point, and unnecessary so far. I don't have the hardware to run a large full precision model with enough ("working") context to ingest hundreds of thousands of tokens and chat with in real time, meaning a large token throughput, to iterate over those over and over. Plus, the software we have available is still full of holes here and there like llama.cpp not properly supporting MLA or SWA. As I said, we'll get there, but it will take a little while more.

>>7958 hey look at that I got my wish

>>8203 >>8205 A serious retard would be able to samefag easily, take p' as an example. This only affects the little guy who occasionally shitposts, and it fosters a culture of elitism and snowflakery, filtering out people who the board doesn't want to see, which ends up creating an echo chamber. All in all, IDs are the pinnacle of reddit.

>>8210 >I WANNA REPLY TO MYSELF REEEEEEEEEEEEEEEEEEEEEEEEEE cry more nigger

>>8211 haha you tell him anon

>>8211 so true and smart and funny

>>8211 with a massive cock too

>>8211 based

>>8210 You're not on 4chan anymore, janny. Get over it.

>>8207 Well I don't know what you're doing but vllm works well even with 8B llama AWQ quantized models

https://github.com/lmganon16/koboldcpp-shared-expetras added --override_tensor you can use this to force full offload of all non shared experts to cpu, put --gpulayers 100 and enjoy a big performance increase tested on RTX 3060 12GB/64GB DDR4: LLAMA_CUBLAS=1 make -j12 python koboldcpp.py --gpulayers 100 --contextsize 8192 --threads 6 --blasthreads 12 --flashattention --quantkv 1 --model ~/TND/models/L4/Llama-4-Scout-17B-16E-Instruct-UD-Q4_K_XL-00001-of-00002.gguf --nommap --override_tensor "([2-9]|[1-9][0-9])\.ffn_.*_exps\.=CPU" command explained: all tensors (layer 2-99 instead of all so that a bit more goes to vram, therefore you have more ram for DE/VMs/programs) that are non-shared get offloaded to RAM. which means if you put --gpulayers 100 all shared experts get offloaded to gpu, which increases T/s considerably (4t/s => 8t/s) >usecase? llama.cpp server is limited in functionality

https://github.com/lmganon16/koboldcpp-shared-expetras added --override_tensor you can use this to force full offload of all non shared experts to cpu, put --gpulayers 100 and enjoy a big performance increase tested on RTX 3060 12GB/64GB DDR4: LLAMA_CUBLAS=1 make -j12 python koboldcpp.py --gpulayers 100 --contextsize 8192 --threads 6 --blasthreads 12 --flashattention --quantkv 1 --model ~/TND/models/L4/Llama-4-Scout-17B-16E-Instruct-UD-Q4_K_XL-00001-of-00002.gguf --nommap --override_tensor "([2-9]|[1-9][0-9])\.ffn_.*_exps\.=CPU" command explained: all tensors (layer 2-99 instead of all so that a bit more goes to vram, therefore you have more ram for DE/VMs/programs) that are non-shared get offloaded to RAM. which means if you put --gpulayers 100 all shared experts get offloaded to gpu, which increases T/s considerably (4t/s => 8t/s) >usecase? llama.cpp server is limited in functionality

>>8206 Yes, didn't mention it, but I'm also not a fan of that. I can often start writing a message, have something more important to do and finish it later, I don't want people to see the half-baked messages.

>>8223 >petra software Nah.

>>8224 how do i delete a post lmao

>>8230 Clock the little arrow to the left of Anonymous

>>7965 >>7975 I stand corrected, they are now posting here, I guess they migrated from 4chan.gay.

Anyone actually get UI-TARS up and running? Curious about the 1.5 release.

>>8222 I doubt a quantized 8b model will be able to consistently read and output so many tokens, reason over them, format them, add to them with PDFs and such, etc. Granted, I didn't try, but still. Even that would probably be too slow for what I'm doing. Hell, it takes around 5 minutes at over 15t/s to output the several 64k token chunks for each iteration. No, this kind of work is not yet something you can feasibly do with local models, assuming modest hardware, I think.

>>8226 lol, imagine being this insecure

what exactly should I be changing these settings to in sillytavern? I have 16GB VRAM (5080) I assume these tokens are set wrong? 400

>>8261 Just set temp to 1 and you're good to go

>>8261 model, rest of setup, OS, full ST export?

>>8258 It disincentivizes writing thought-out posts and puts peer pressure on sending messages immediately without even reviewing them for spelling mistakes. >>8261 Just run the program, what's the issue?

>>>8261 who let the techsupport beggars in?

>IDs Remove this shit

>>8267 Fuck off

>>8125 >constant back and forth changes Now that you mention it, the maybe it really is the feds.

>>8261 That should work. Maybe change the Response (amount of tokens model will output) and see if the Context is set the same as is set in whatever backend/loader you are using. Sometimes you have a model with, say, 128k tokens of context window, but due to memory limitations you set the limit to just 16k, you have to set that same 16k number in that Context slider/field.

>>8265 In other words, it's a incentive to make more honest posts. Sounds like a good thing, I'm sure that also makes anon think two times before acting like jerks, so that's another plus. >>8267 you tell them anon

>>8278 just go back to reddit anon, you will feel at home.

>>8278 Atta boy.

>>8278 Thanks bro

>>8278 Thanks bro

>>8279 >>8288 The duality of anon

>>8263 thanks, will do that! >>8264 I'm using Mistral-Nemo-Instruct-2407-Q6_K_L as my model (I was told to use this for my 16GB card) Also one issue I have is that the characters tend to write massive blogs on text and "advance" the RP too rapidly multiple steps at a time. How can I reduce how much they will write each time they generate text so it goes at a slower pace? Like if I say "let's go to the beach" they will write a blog about saying yes, then going to said beach, arriving, setting up towels and then ending their text their. It's like holy shit let me respond. I'm sure it's a setting I can change to lower how much they respond back.

>>8321 Telling the model to slow down after it has already started writing blogs is no use. You have to keep an eye out for that at the beginning of the RP. After few medium-length replies it picks up on it and keeps the pace mostly the same.

>>8327 >Telling the model to slow down after it has already started writing blogs is no use. So it's not an actual setting in ST? I just have to start the RP with something like >Hello character (keep replies short-length) that? I don't mind starting it over I was mostly testing, but kinda figured reply length was some setting in ST.

>>8333 You can put that at the end of your system prompt for starters

>>8333 You can set a max token output limit, but it doesn't affect what the model wants to write. What happens is that it'll try to write another blog, but gets cut off at token 400 or whatever you'll set.

>>8333 You can set the response length in tokens but it will usually just cut off the response, so it's better to tell the model what you want instead. You can also put instructions like that in the system prompt section of the settings instead of putting them in your replies, it might work better (or worse)

>>8348 >>8343 >>8338 awesome, thanks fellas

>>8321 use mag ms sloptune posted somewhere ITT, pronably should use iq4_xs to fit it in vram fully theres also master preset somewhere on catbox also ITT

>wake up >everything is up in flames Man.

>>8258 It's not really about being insecure. Having people see and answer half-baked posts only reduce the quality of a discussion.

>>8375 first time?

>>8375 12vhpwr was a mistake

>>8398 >12vhpwr I don't know why they still push this meme.

Gemma 3 27B q4 QAT is pretty good at writing ENF stories and coming up with scenarios based on images/text prompt. The disclaimers are funny because I'm pretty sure they're like a self fulfilling prophecy that encourages more lewd content. I think I'm all set lads.

Half the thread is offtopic about post "quality" already. And a bunch of faggots praising the ids because thread is finally becoming more like reddit. It is a good thong every single one of you maladjusted retards got bullied. And you should really just kill yourselves nos. Words fail to describe how much limp wristed faggotry is condensed here. Half of you probably take more estrogen than the average 4chan janny.

>>8428 I need to play around with it too. What are you using to run it with text + img?

>>8428 From what I've tested and seen, QAT seems to be good at q4 and below, if quanted to higher quants it seems to be worse than regular. I wonder why.

>>8428 Using any jailbreak prompt? Never had good experiences with anything Gemma

>>8375 Shouldn't have bought intel

>>8442 Lmao

>>8431 KoboldCPP has the easiest support for images. If you have the vram, exllama will run it too. Then just sillytavern and chat completions.

>tried deepseek v3 0324 >mfw I for one will be very welcoming to the Chinese century.

>>8449 You mean proper native support for the model or that thing that turns the image into a promp of sorts?

>>8430 Pretty much this.

>>8459 same, I'm not going back to 70b tunes despite them running 3-4 times faster on my rig (5-7t/s vs 1-2t/s)

>>8459 V3 0324 is incredibly cucked when compared to the old version tho

>>8449 Wait what, since when does Exllama have image support?

>>8398 >He didn't powerlimit his GPU

>>8505 >recieve Ok ESL

>>8555 Since people begged turboderp to add qwen VL. It works with pixtral as well. Your options are VLLM, transformers, exl2, kcpp and obama. I hate obama.

>>8197 gemma3 q4 running on one 3090 can be pretty good. Do you really need more? Depending on the type of data, you may make the process more robust with some pre/post-processing.

>>8423 A small connector for a small PCB. While the obvious choice of two thick wires is more expensive and less flexible than a pack of thin wires, it is an okay connector in theory—just its actual implementation is poorly engineered.

>check 4gay >its active are you serious

>>8766 That OP really fucked it up. Retards are drawn to it like flies are drawn to a fresh pile of shit.

What is the situation with Deepseek, MLA, ktansformers, and Unsloth? Does ktransformers support mla? Do I need new quants for that? Will unsloth release updated magical quants? I'm RAMlet with 96VRAM+256RAM

Leto just wiped most of 4gay while claiming he had an in person FBI visit. Was posting the 4gay url on kiwifarms worth it?

Qwen promised to release the model in April, right? Surely this is the week.

https://github.com/Tencent/InstantCharacter Consistently generating new pic evry message would be peak rp

I might actually shit myself.

>>8791 What are they waiting for? R2 could come out anytime and make every other model obsolete. If Qwen3 weren’t shit, they would have released it already

>>8789 ktransformers should but my P40 shitbox got filtered by flash attention ikllama.cpp has mla + fa and works with existing quants, but the server is shit, doesn't support jinja, and looks like it hasn't been touched in 5 months. Unless my gguf file is broken it's also messing up the chat template for R1 so you need to fuck about with text completion mode llama.cpp main got jukofyork'd and still wastes vram for no real speed-up, so pretty much unusable grim

>>8802 Why did they let this happen when both ik_ and ktransformers have working implementations and both are based on old-ass versions of llama.cpp?

Is this the official /lmg/ now? I guess CUDA dev Anon and summary bot Anon will be the final seal of approval.

>>8806 Some drama ass shit and bad blood. They hate each other and have a beef about giving credits for contributions

>>8806 No one really knows, here's the bickering that took place when someone tried to move a feature from ikllama.cpp to llama.cpp https://github.com/ikawrakow/ik_llama.cpp/discussions/316 Also saw this while searching for the above link kek https://github.com/ikawrakow/ik_llama.cpp/discussions/319

►Recent Highlights from the Previous Thread: https://meta.4chan.gay/tech/67288 https://files.catbox.moe/u4jlh8.zip ►Recent Highlight Posts from the Previous Thread: https://pastebin.com/YTXUbc3Q Why?: 9 reply limit >102478518 Fix: https://rentry.org/lmg-recap-script

>>8809 8chan went down right when I went to post. I was able to get the regular script to work with gaychan over the weekend. Turns out they embed the initial json of the thread into a script tag at the bottom of the html. The script ran for 4 hours and 6 minutes. The final recap is 9328 characters long. Of course, right when I get it working the retard admin nukes the site. So I guess instead you can have this first ever weekly /lmg/ magazine. It only covers until Saturday night (right before rocket migu), the images are only thumbnails, and I did no proofreading.

>>8815 stay here honey, don't leave us

>>8815 Thank you Recap Miku

Other than the single twitter post, was there any official communication from 4chan anywhere?

>>8815 >>8814 based. i kneel

>>8820 There was a screenshot of an email that they supposedly sent to the jannies but it's 50-50 whether it's fake

>>8814 i have to say, this is a very good recap great format aswell

>>8802 I tried IK for non deepseek expecting it to be faster. It wasn't. Even for CPU only.

rumors of 24gb intel arc... https://videocardz.com/newz/sparkle-confirms-arc-battlemage-gpu-with-24gb-memory-slated-for-may-june would you risk it?

>>8833 IK was forked like over half a year ago, so probably improvements made there aren't enough to catch up to the upstream master branch. Now if only someone could combine them.

>>8835 Whenever anyone combines them, IK himself screeches.

>>8834 I wouldn't consider any card under 48GB at this point.

>>8834 I already have 4x3090 so no point. If I was starting over then maybe. Once you go nvidia/amd/intel you can't go back if you want them to stack.

>>8814 >>8815 i have a slightly more recent archive (a few posts after rocket migu) i modified the css hrefs, thumbnail hrefs and flags for viewing pleasure: https://files.catbox.moe/p4t8g9.7z >>8834 100% if not over 400$, since i can get used 3090s for around 600$ here

>>8838 retarded question, but you can't split across different gpus via vulkan or something? I bought a 3070 a long time ago before the joys of AI

>>8834 half the memory bandwidth of a 3090 if the bus width is the same as the B580. if it's priced reasonably then the desperate who want to generate text will grab it, even if the software compatibility is lacking. i'd rather add a third 3090 or 4090 if i was getting another card.

>>8840 you can with llama.cpp vulkan if not, you could use one gpu for tts/imagegen/summary LLM/whatever and other one for erp LLMs

>>8842 Vulkan despite best efforts is quite slow. I do agree it can make a decent bonus GPU. If 2080ti 22g is cheaper than this card it's still a non-starter. Whatever you lose to the older cuda version is going to be infinitely better than ipex.

>>8844 2080ti 22g now costs 600-700$

https://nitter.net/8chan_se/status/1913554775540486357 uhh anons.. 8chan bans countries.. uh oh

>>8854 as long as they allow vpn posting they'll never stop the spam anyway

>>8854 Good

good mornin' fellow redditors, I hope we have an heckin' great day today. >>8814 >

>>8854 cringe

>>8863 what the f

>>8863 seems about right. >t. russian

>>8854 Now they just need to ban India and we'll all have a happy Easter

Reposting this banger https://suno.com/song/4e90afb1-78db-4157-a2db-fd1626934b54?sh=UkNe0N539mqGfB65

https://huggingface.co/collections/Skywork/skyreels-v2-6801b1b93df627d441d0d0d9 skyreels v2, finetune of wan https://huggingface.co/Skywork/SkyReels-V2-I2V-1.3B-540P 1.3b i2v aswell

>>8896 5B model coming soon

wintoddlers.. not like this https://www.techpowerup.com/forums/threads/nvidia-576-02-breaks-gpu-temperature-and-rpm-sensor-read-out-and-use-for-some-applications.335664/ https://www.nvidia.com/en-us/geforce/forums/game-ready-drivers/13/563010/geforce-grd-57602-feedback-thread-released-41625/

>>8898 I never update nvidia drivers though

some videos made by skyreelsv2

>>8896 I can run wan I2V 14B 720P on my 3090 but this shit says "Generating a 540P video using the 1.3B model requires approximately 14.7GB peak VRAM, while the same resolution video using the 14B model demands around 43.4GB peak VRAM."

>>8839 Thank you. It's good to have a complete archive. Shame the full images are lost.

>>8896 >0.2% improvement in benchmarks Why does it exist?

>>8902 anon thats with no quant, no offloading cache, and no optimizations and likely with prompt enhancer >>8908 see picrel, i2v much better

>>8894 I like this one much better https://vocaroo.com/11DnjSpKngGn

>>8910 >Note the peak memory of GPU is 64G+ if use --prompt_enhancer I'll wait for real examples, theirs seem cherrypicked af

>>8912 YuE? slop otherwise

>>8914 qwen32b IQ4_XS is 16gb, can be run on 100% cpu aswell or offloaded partially would be great if benchmarks are to be believed

>>8915 Not related, but I found a SNES music generator under MIT: https://github.com/parlance-zz/dualdiffusion Samples: https://www.g-diffuser.com/dualdiffusion/

>>8920 https://github.com/jthickstun/anticipation/ https://github.com/1aienthusiast/Anticipation-WebUI

>>8920 >trianing an audio diffusion model on glorified midi ytho. Wouldn't you be better off training an auto-regressive textgen model on pairs of midi instructions + song descriptions and then running the output through the wavetable?

>>8901 I wanna love video models, but minutes or even 1/2 hour to make a short clip kills my motivation to run them. Plus no multi-gpu and many speedups needing 4090 or better.

>>8925 This can make you a short clip in a few minutes: https://github.com/Lightricks/ComfyUI-LTXVideo also this: https://github.com/lllyasviel/FramePack

>>8927 framepack takes 20 mins on 3060 with sageattn and default settings that anon should check ltxvideo def tho he should also try the 1.3b i2v skyworks model >>8925 >>8925 >>8925 >>8925 >>8925 >>8925 >>8925 heres a few yous

>>8927 Thought LTX was kind of weak compared to wan, hunyuan, etc. Maybe this skywork model will be better. It's not like image where it can go along with your chat or TTS. Mainly useful for ha ha memes or assembling a long video for public consumption. Only benny is to do it just to do it in my case.

>>8934 the new version is pretty good

>>8935 Guess I will see.. I literally downloaded a bunch of these promising myself I would try them and then fucked around with chatbots instead.

>Seeing over-priced 'premium' motherboards with 'AI ready' hype marketing all over them A few months ago I would have thoroughly rebuked anyone crying about AI hype marketing but we've reached a point where I have to admit they are now correct. Like what the fuck makes a motherboard 'ai ready'? It has a PCIE slot? wow. My 5 year old AM4 motherboard is AI ready too I guess.

ComfyUI and it's consequences have been a disaster for local models, I want simple frontends back

>>8951 Yeah, especially if it's shit like this. A standard consumer Ryzen gayman board with dual channel memory and two slots (x16 + x8) for $700 + tip.

>>8954 But it has random paper thin, aluminum sheets with black paint on them. And look at that greebling...I mean uh.. THERMAL STRIATIONS

>>8954 It can run Nemo so it's totally an AI machine.

Is this one of you guys trolling reddit? >Im just new to all of this, so I am not sure which models to install with ollama. >Here are my pc specs: >RAM: 32GB GSKILL TRIDENT Z - 6400MHZ >CPU: I7 13700K - Base Clock >GPU: NVIDIA 4090 FE - 24GB VRAM Like the fucking brand name of the RAM and the GPU are even remotely relevant. Oh... sorry sweaty... If it were a Hyper X Fury you could run R1 but unfortunately G.SKILL TRIDENT maxes out at Phi-2 Medium.

>>8953 no

>>8951 >It's not AI if my computer doesn't heat my house.

>>8960 You might not like to hear it, but it's the truth. I hate this tendency of freetards to make everything difficult because they don't have to cater to average users and only enthusiasts. When I want to generate something inconsequential with my local model, I don't want to think about which nodes to connect or go out of my way to search for "workflows" for fuck's sake. that's why closed source is always superior, they do it for the money and they know it wouldn't fly if people had to pay for it. Don't get me wrong tho, I don't mean that comfyui is a bad thing, it's very powerful and all, but I wish they had tried harder to make it less annoying to use for things that are less involved.

>>8967 but comfyui is easy to use, just do a bit of basic stuff and thats it i get that normies wont be able to really use it but it isnt that hard..

>>8967 decent quality bait

>>8958 >He doesn't flex his AI-ready machine

>>8982 This just gave me an idea. I should just buy up a old office PC cases, slap "AI Ready" decals on them and then resell them for like 200 dollars each.

So what RP model you guys recommend? How's Aion-RP-Llama-3.1-8B-f16.gguf ? I wanna create some nsfw stories involving Lara and her hairy puss.

>>8990 post gpu, ram, cpu, OS, frontend You're using, age, sex, race

>>8990 It gets +100 reddit karma if you run it on Trident G SKILL Z memory.

>>8953 It's the opposite. I think textgen is still behind imagegen because it lacks a standard node-based editor.

>>8994 What would you even put in the nodes?

HAPPENING! meta.4chan.gay PROVEN TO BE A HONEYPOT HAPPENING!!!!!!!

>>8991 3090, 32gb ram, AMD Ryzen 9 7900 No idea how to run these models, new to the scene

>>8998 post the rest if trips

>>8994 language is linear. There's literally no way to 'node'-ify it.

>>9000 digits of truth..

>>8997 Not at all surprised

How do I get this to run, following the rentry guide Guess the local host is wrong? Or do I still have to add the api keys from the aion site?

>>8996 >>9002 Samplers, loras, control vectors, RAG, tool calling, building up prompts in phases, output processing, multi-step requests, etc. Most of it would be linear, yes; but I think having a standard way to define workflows would open up a lot of possibilites that would be a lot easier than writing Python boilerplate. Off the top of my head some simple examples I've thought about are to have a reasoning model use a different temperature for the thinking block vs the output, feed a response back into a model some amount of times to iterate on the response, or to generate many responses and have a final aggregation prompt. You would think with agents being the latest fad, there would be more interest.

>>9025 >>9026 Forgot to hit Launch lol

>>9029 There are workflows for agents in proprietary softwares, but not with that granularity. You'll need to find another comfyanon autist to pull that off. I can't even imagine the amount of work when the backends can't already keep up with the new stuff coming out

>>8997 >place that spams CP by the second is a honey pot Shocker

>>8997 >>9050 It's pretty much irrelevant because you can post through Tor and any other proxy, they barely ban anything, same when they were offering the 4chan proxy too.

Seems like it gets stuck when I press continue? New to this Wat do pls?

https://yummy-fir-7a4.notion.site/dia Babe, wake up, new TTS just dropped.

>>9064 this could be fun

>>8958 Perhaps they just copy and pasted it. NFW they typed all that out.

>>9063 install linux

>>9063 Well, you've triggered a stop sequence

>>9090 meaning? how do I resume?

>>9094 by installing linux mint 22.1 its very easy to install

>>9064 >Play with a larger version of Dia: generate fun conversations, remix content, and share with friends. 🔮 Join the waitlist for early access.

>>9094 Honestly I've never run into problems with stop sequences, usually it's just plain refusals. You should be able to configure them in ST

>>9102 no cloning, no use

>>9064 You have no control over the tone of voice even with an audio prompt. In most gens it just gets very angry. >>9187 Cloning works by continuing from an existing clip. It's literally one of the examples.

>>9187 Seems like they're just learning from sesame, give away a shitty 1B model and sell a service for your bigger model

Have there been any decent models that fit on 24gb released in the past week since 4chan's been down? Haven't been keeping up with the threads since then

>>9195 we're stuck on the same sloppa anon

What is the best inpainting model nowadays? Also why are half of the replies non related to AI

>>9195 GLM-Z1 was the only thing interesting. Nobody uploaded any decent quants.

I'm always surprised by the expressivity of gptsovits. If it was a bit more polished it'd be elevenlabs-tier. https://voca.ro/1ntiLiusbWpN

https://github.com/SandAI-org/Magi-1 https://huggingface.co/sand-ai/MAGI-1 >The first autoregressive video model with top-tier quality output https://xcancel.com/SandAI_HQ/status/1914303284954996749 China won... again...

>>9235 Videogen is eating good, I lost count of the number of models we got this month alone

>>9235 But can it do porn?

>>9235 too dumb to understand the math from the tech report, it's so over

>>9235 how to stop being poor

>>8068 how do you masturbate with an llm? i use mine to generate short stories that go along with images i make in comfy >ice agent gardevoir rounding up mexicans (to fuck)

>>9254 not him, but my routine is usually like this: I come up with an interesting idea, spend an hour crafting a card and figuring out every model quirk to make it work, then fap on to something else, never using this card again

>>9235 holy based.. i kneel

>>9195 Gemma 3 27B got a QAT version optimized for 4bit. Supposedly it's very smart, but I can't get it to do good RP because its prose is drier than a desert and it's just as horny. The lobotomy is deep as well. Interested to see if any finetunes can save it, but they would probably undo the QAT magic. So far Mistral 24B is still undefeated despite its repeating issues. But I'm interested if anyone has managed to get good results from Gemma or QwQ somehow.

>>9215 I tried GLM-4 and Z1, they are so slopped it's unreal. Synthetic data shows in every sentence, some phrases that I haven't seen in a year popped up. Yes, it's probably broken or something in llama.cpp, but it loaded perfectly and works, outputs everything; hence why I think it's just shit.

>mogao turned out to be closed source its over >https://artificialanalysis.ai/text-to-image/arena?tab=leaderboard >sneed

Noob here, I've been enjoying generating stories with Gemma abliterated, are there any other uncensored models worth trying for stories/RP? I can only run around 32B max.

>>9290 The ones I've seen people usually recommend/shill these days are Rocinante, Nemo, and Cydonia

>late april >the only nothing the entire year besides Deepseek, severely undertrained LLaMA4 and the usual worthless 1~32b scraps companies give out Where the fuck are the open flagship models? Deepseek is literally the only good thing we've seen all year.

>>8951 It's double the price. Anyone that notices is not ready for AI.

>>6258 https://github.com/JohannesGaessler/elo_hellm/issues/2 >Interrogation-based game à la Inhuman Conditions >Inhuman Conditions is a game in which one person is an investigator and one person is a suspect. The investigator wins by correctly determining whether the suspect is a human or a robot. The suspect always wins by being identified as a human. So if the suspect is a human, both players are on the same team; if a robot they are on opposing teams. The investigator asks questions that the suspect answers. A human answers in a normal way. A robot either has restrictions on what they can say or they have a compulsion to include something weird. >For this project the game concept could be adapted to have model A roleplay as either some character or as a robot/demon/alien pretending to be said character. Model A then roleplays some interaction with model B. If model A is roleplaying as an impostor then wins/losses can be used directly for Elo ratings. If model A is roleplaying as a human then the models are effectively playing against a benchmark. Models should not always play against each other because otherwise model B is being rewarded for a bias towards labeling model A as an impostor. If model A is an impostor it only wins if it can fool model B while fulfilling some constraint. It will be necessary to use a model as a judge to rule whether model A is complying.

>>9348 >CUDAdev turns convincing RP into a benchmark so that all the companies will have to train on RP to benchmaxx it Absolute madman

>>8809 I would be fine with this site.

>>8809 >>9423 https://raw.githubusercontent.com/JohannesGaessler/JohannesGaessler/refs/heads/master/README.md A bit annoying that I will potentially have to reset my IP to avoid retroactively de-anonymizing myself but I guess it's less annoying than the 4chan.gay admin.

>>9337 Llama 4 got its image/audio capabilities removed, got safetymaxxed, finetuned on mostly Llama 3 datasets (it has the same annoying quirks). The models we've got are either an early "maximum-compliance" training run, or something that got hastily retrained at the last minute due to legal concerns. DeepSeek R1/V3 are much more undertrained in comparison... that's not the issue with Llama 4.

>>9424 At least you can use VPN not tied to actual billing details unlike the old site.

>>9437 They totally intended to train their 400b model for half the time and tokens of their 100b one, yes?

>>9442 It still supposedly got trained for 22T tokens; it's not like the other 20T tokens would have made it immensely better, seeing how sub-par Scout is. For comparison, DeepSeek R1 (685B parameters) was trained on about 15T tokens and it even has twice the number of routed experts per layer (256 instead of 128) than Llama 4 Maverick.

>>9235 Anyone made any test gens with this yet?

>>9424 IDs disincentivize shitposting. Terrible, isn't it? >>8033 >>8142

>>9455 Oh I just installed everything to realize that they didn't even make the 4.5B model weights available. Classy. also got approved for blt-7 repo, although as always with non hf-ified meta models its dependent on meta's garbage in-house code so I'll probably just go back to bed and pretend it never happened before I get it working without using hugging-shit's shitty model downloader that just assumes you aren't running with an OS drive already filled to the brim with python shit.

>>9507 >that just assumes you aren't running with an OS drive already filled to the brim with python shit. Just like ollama.

>>9507 >hugging-shit's shitty model downloader export HF_HOME yourdir >OS drive already filled to the brim with python shit In project's dir: python -m venv yourvenv source yourvenv/bin/activate After you've done with installing: python -m pip cache purge

>>9506 I made neither of those posts, if that is what you're trying to insinuate. I made the post about Elo HeLLM in the other thread before it got nuked, someone else then copied it to here.

>>9518 I already figured out a workaround. But in either case it's another episode of additional troubleshooting required. But I'm done. It's just going to be more metaslop at best. >inb4 hello sars here is how to redeem the shared experts anon shows up and takes exception.

GPTSoVits v4 was released a few hours ago https://github.com/RVC-Boss/GPT-SoVITS The improvement over v3 is basically this: Version 4 fixes the issue of metallic artifacts in Version 3 caused by non-integer multiple upsampling, and natively outputs 48k audio to prevent muffled sound (whereas Version 3 only natively outputs 24k audio).

>>9518 venv are really fun until you want to move the project folder. I guess if you don't have poorboy internet, downloading 15gb of pyshits every time doesn't matter.

>>9562 Use conda then

>>9562 Without venv, projects engage in a battle royale over library versions

>>9519 Not insinuating, just heavily implying

>>9578 No one cares retard, try to contribute to the thread instead

One week until LLaMA 4.1 and Behemoth.

>>9567 I have a conda for 11.8 and 12.6, 99% of projects work in them. Devs with main character syndrome pin dependencies to arbitrary versions and their setup scripts try to shit things up for the sake of newbies. I just use --no-deps and fill in whatever is missing. Also slightly lower chance to be fucked by requirements.txt with compromised packages.

>>9601 Placing my bets on Behemoth being API only.

>>9602 Back to SD1.5 are we?

>>9601 Is there even the slightest chance that this won't be a massive flop?

>>9606 Reasoning scout/maverick not being super retarded and making up for the 17b active parameters, but who are we kidding.

>>9606 no

>>9605 I like certain slop. XL+ is soulless

>>9606 Behemoth 2T/288B-A is going to beat Deepseek R1 (but not R2)

>>9613 Behemoth is probably the same benchmaxxed garbage as 405B was.

I just ordered one of those chink GMK EVO-X2 computers (since the framework desktop won't be out for 6 more months and the HP Z2 Mini G1a is 4500 dollars). How many days/hours will I be able to use it before something breaks?

>>9601 >LLaMA 4.1 *If* that's coming out so soon, I bet Meta is going to double down on: >I can't help with that. I still can't believe Gemma 3 Instruct ended being less censored than Llama 4, as long as you provide a good prompt. Llama 4 effectively killed off lolisho conversational roleplay/ageplay, if you make any reference to ages. Even just discussion along those lines is off-limits to the models.

>>9650 >if you make any reference to ages. Even just discussion along those lines is off-limits to the models. If I didn't know any better, I would say the models they put out took all requirements for copyright, safety, and carbon emission to the extreme as an example of how detrimental they are for the government to ease up on them. Sad truth is they just belong to the cult of safety.

https://huggingface.co/nari-labs/Dia-1.6B they released the model

>>9656 They trained it with multiple speakers, so the voice cloning is crap.

>>9653 I don't know if they jacked up 'safety' just to make a point that it's harmful to model performance. It feels like they targeted hard use cases that aren't benchmark-relevant and that may cause public embarrassment (as well as being politically currently very unfavorable) and I have reasons to suspect they didn't initially plan to go so hard in the final models. The anonymous Llama 4 models served on LMArena on late March really seemed cunny-friendly, even though their system prompt almost certainly didn't have anything in that regard. I still wonder if the Llama team panicked when they saw what people were sending and what their models were responding. I distinctly remember that at some point Meta began filtering user inputs (containing ages, for example) to their models, beyond what LMSys was doing on their side.

>>9666 The guy who thought that serving a novel-sized response by default to all prompts was a good idea should be shot

>>9682 Definitely not for everyone or every prompt. You could tell it to output shorter responses and it would comply, to be fair. Responding with context-appropriate dynamic length to user inputs is something that LLMs in general seem incapable of doing reliably without serious hand-holding, in any case. They tend to lock to the most prominent patterns in context.

>furk got fucked deserved

EasyEdit2: An Easy-to-use Steering Framework for Editing Large Language Models https://arxiv.org/abs/2504.15133 >In this paper, we introduce EasyEdit2, a framework designed to enable plug-and-play adjustability for controlling Large Language Model (LLM) behaviors. EasyEdit2 supports a wide range of test-time interventions, including safety, sentiment, personality, reasoning patterns, factuality, and language features. Unlike its predecessor, EasyEdit2 features a new architecture specifically designed for seamless model steering. It comprises key modules such as the steering vector generator and the steering vector applier, which enable automatic generation and application of steering vectors to influence the model's behavior without modifying its parameters. One of the main advantages of EasyEdit2 is its ease of use-users do not need extensive technical knowledge. With just a single example, they can effectively guide and adjust the model's responses, making precise control both accessible and efficient. Empirically, we report model steering performance across different LLMs, demonstrating the effectiveness of these techniques. https://github.com/zjunlp/EasyEdit https://zjunlp.github.io/project/EasyEdit2/ also https://github.com/nari-labs/dia https://huggingface.co/nari-labs/Dia-1.6B >Dia is a 1.6B parameter text to speech model created by Nari Labs. >Voice cloning. See example/voice_clone.py for more information. https://huggingface.co/spaces/nari-labs/Dia-1.6B

Better Estimation of the KL Divergence Between Language Models https://arxiv.org/abs/2504.10637 >Estimating the Kullback--Leibler (KL) divergence between language models has many applications, e.g., reinforcement learning from human feedback (RLHF), interpretability, and knowledge distillation. However, computing the exact KL divergence between two arbitrary language models is intractable. Thus, practitioners often resort to the use of sampling-based estimators. While it is easy to fashion a simple Monte Carlo (MC) estimator that provides an unbiased estimate of the KL divergence between language models, this estimator notoriously suffers from high variance, and can even result in a negative estimate of the KL divergence, a non-negative quantity. In this paper, we introduce a Rao--Blackwellized estimator that is also unbiased and provably has variance less than or equal to that of the standard Monte Carlo estimator. In an empirical study on sentiment-controlled fine-tuning, we show that our estimator provides more stable KL estimates and reduces variance substantially in practice. Additionally, we derive an analogous Rao--Blackwellized estimator of the gradient of the KL divergence, which leads to more stable training and produces models that more frequently appear on the Pareto frontier of reward vs. KL compared to the ones trained with the MC estimator of the gradient. iirc I wanted to post this for Johannes and rereading it yeah I think it was this paper. oh 70% Size, 100% Accuracy: Lossless LLM Compression for Efficient GPU Inference via Dynamic-Length Float https://arxiv.org/abs/2504.11651 >Large Language Models (LLMs) have grown rapidly in size, creating significant challenges for efficient deployment on resource-constrained hardware. In this paper, we introduce Dynamic-Length Float (DFloat11), a lossless compression framework that reduces LLM size by 30% while preserving outputs that are bit-for-bit identical to the original model. DFloat11 is motivated by the low entropy in the BFloat16 weight representation of LLMs, which reveals significant inefficiency in existing storage format. By applying entropy coding, DFloat11 assigns dynamic-length encodings to weights based on frequency, achieving near information-optimal compression without any loss of precision. To facilitate efficient inference with dynamic-length encodings, we develop a custom GPU kernel for fast online decompression. Our design incorporates the following: (i) decomposition of memory-intensive lookup tables (LUTs) into compact LUTs that fit in GPU SRAM, (ii) a two-phase kernel for coordinating thread read/write positions using lightweight auxiliary variables, and (iii) transformer-block-level decompression to minimize latency. Experiments on recent models, including Llama-3.1, Qwen-2.5, and Gemma-3, validates our hypothesis that DFloat11 achieves around 30% model size reduction while preserving bit-for-bit exact outputs. Compared to a potential alternative of offloading parts of an uncompressed model to the CPU to meet memory constraints, DFloat11 achieves 1.9-38.8x higher throughput in token generation. With a fixed GPU memory budget, DFloat11 enables 5.3-13.17x longer context lengths than uncompressed models. Notably, our method enables lossless inference of Llama-3.1-405B, an 810GB model, on a single node equipped with 8x80GB GPUs. https://github.com/LeanModels/DFloat11 word count per post is more than 2k. Cool. anyway https://greasyfork.org/en/scripts/533067-fullchan-x https://greasyfork.org/en/scripts/533169-lynxchan-extended-minus-minus using these scripts if anyone wants something more than the default

>>9606 >>9704 >Notably, our method enables lossless inference of Llama-3.1-405B, an 810GB model, on a single node equipped with 8x80GB GPUs Yeah but 405b Q8_K fits on only 6 of those GPUs while being essentially lossless

>>9697 They suspended me for spam because I commented on a PR. It took 2 months to get reinstated. As much of a jizzer as he is, I bet he did nothing.

>>9704 Thank you for the paper on KL divergence. If I understand the paper correctly they suggest determining the variance of the KL divergence between models from the values per token instead of the values per prompt/chunk of text. That is already how the variance of the KL divergence is being estimated in llama-perplexity.

>>9704 >>9733 Actually, I think I need to retract my previous post. Looking at the notation again what I think they're suggesting is to calculate variances per token position instead of calculating one variance for all token positions. In the context of Monte Carlo methods in physics I have seen this technique under the name "stratified sampling". For llama-perlexity I think it's not really worthwhile to implement since the variance is already so small (compared to the bias of which text you use as input). But I'll definitely remember this for training.

Anyone do any kind of systems or automation with LLMs as opposed to just pure chatting? I think it would be cool to hook up various personal programs and home automation things to a LLM so that I can tell it to do things. I've been thinking of using open-webui as kind of the core runner and api provider. I think I would just create and make available a whole ton of tool calls or use MCP, which as far as I can tell is basically just a format for tool calls. I don't know if it would be necessary to do like, sub-routing to different models or anything. Thoughts? Anyone work on anything similar?

TTRL: Test-Time Reinforcement Learning https://arxiv.org/abs/2504.16084 >This paper investigates Reinforcement Learning (RL) on data without explicit labels for reasoning tasks in Large Language Models (LLMs). The core challenge of the problem is reward estimation during inference while not having access to ground-truth information. While this setting appears elusive, we find that common practices in Test-Time Scaling (TTS), such as majority voting, yield surprisingly effective rewards suitable for driving RL training. In this work, we introduce Test-Time Reinforcement Learning (TTRL), a novel method for training LLMs using RL on unlabeled data. TTRL enables self-evolution of LLMs by utilizing the priors in the pre-trained models. Our experiments demonstrate that TTRL consistently improves performance across a variety of tasks and models. Notably, TTRL boosts the pass@1 performance of Qwen-2.5-Math-7B by approximately 159% on the AIME 2024 with only unlabeled test data. Furthermore, although TTRL is only supervised by the Maj@N metric, TTRL has demonstrated performance to consistently surpass the upper limit of the initial model, and approach the performance of models trained directly on test data with ground-truth labels. Our experimental findings validate the general effectiveness of TTRL across various tasks, and highlight TTRL's potential for broader tasks and domains. https://github.com/PRIME-RL/TTRL pretty interesting

>>9888 I'm also interested in this but all I do is browse imageboards and fap, what is there to automate? Maybe if I had some cameras and a dev board I could automate some kind of AI powered security system.

>>9901 >what is there to automate? E-stim, vibrators or sex machines via XToys.

Cache quantization was just added to EXL3. It's actually worth using now. https://github.com/turboderp-org/exllamav3

>>9941 Is min-p implemented yet? It is literally the only useful sampler

>>9942 Sadly, no

>>9941 Does it support Ampere yet?

>>9945 It always did. Just not as fast as exl2 yet.

>thread here is bleeding out >cunnychan /lmg/ thread is dead Is there another /lmg/ somewhere that I don't know about?

>>10055 sam won...

>>10055 We didn't know how good we had it...

>>10055 There is >erischan.org/aes/thread/1263.html but it's also slow. I think people underestimate how important a flow o randoes that come and go is necessary to keep a general alive. Otherwise it becomes a circlejerk between the same 5 to 10 guys.