Yeah - I think he’s worried that we have a tool that’s almost an oracle that can just get straight to the heart of a given problem.
The problems aren’t always that interesting in their own right. But the quest to solve them is often what drives the invention of new techniques.
We ultimately got the Langlands program and a large amount of algebraic number theory out of attempts to solve fermat’s last theorem. A “clever” approach using techniques from the 1800s wouldn’t have been anywhere near as fruitful for mathematics as a discipline.
AFAIK, the “large” qualifier came when transformers allowed to scale the size of language models compared to the recurrent models that where in fashion before. And although BERT isn't large by today's standard, it was large enough for the time.
idk the definition is fuzzy. thats why people use the "modern" qualifier to talk about decoder-only style and this is also not clean since you now have reasoning models which are separate
To that point, given a corpus of writing from person A and another from person B, I have no doubt it’s easy to train a classifier to determine who wrote what. In fact I believe law enforcement agencies already have these classifiers.
The only difference here is that Anthropic is actively trying to make the watermark undetectable.
They're not trying to make the watermark undetectable, that would defeat the point of a watermark. They're making it detectable, but not make the text obviously watermarked
Out of interest, do Musk's politics impact your decision on whether or not to use Grok? I'd be interested to know where folks lie on the (Agree / Disagree) and (Use / Don't use) axes.
It does. I believe he is one of the worst people and contributed to misery of humanity. I had nothing against him until he dismantled USAID. The richest man in the world did not go to a party on weekend to make sure that poorest men on the world have less help. He was basically on the side of HIV.
So i never use his products.
Beside don't read too much in to benchmarks. They are alrrady ruined by Goldhart principle. Tgese models have already seen most of the data.
Absolutely, if there is any somewhat reasonable alternative, I will always use a non Musk product. Its less about morals but more about self interest. I am from Europe and Musk supports far right extremists and a breakup of the EU. I will not support and enable someone who intends to do me harm.
I disagree with Musk's politics but it does not impact my decision to use Grok. That's because being serious about aligning my capital to my values doesn't leave much in the way of eligible products or services. I consequently decide not to worry about this as a moral axis for my life.
Interesting question. I suppose it comes down to how much you allocate his involvement or presence to a product? I’d imagine Grok is built by hundreds of engineers who are all unique individuals from various backgrounds. If Elon simply “leads” from a very surface level where he has no direct day to day involvement in Grok releases does that make it more palatable? Or is the question really about how involved he is? Or is simply being the leader (even if he was 100% absent and only had his name attached to a project/company) enough to boycott?
On a similar note, how much Elon hate is about his politics vs his trillionaire status vs what I like to call “watercooler hate” where folks simply parrot the loudest opinion in order to be accepted into the group?
On a final note, my son is in primary school and recently brought up in a dinner time discussion that “Elon is really bad” - this is a kid who has no social media (unlike some of his peers who are already on TikTok) and doesn’t watch traditional media.
The man did a nazi salute on live TV, not once but twice. And you know he meant it. What else is there to doubt? Do you think a white supremacist can be a good person?
> The man did a nazi salute on live TV, not once but twice. And you know he meant it.
He's also publicly and militantly supported quite a few far right political parties throughout Europe, particularly those who have a long track record on race-based topics.
its pretty on the record that musk takes a direct role in writing the system prompts, no? and is eager to make updates if whatever ml products arent sufficiently matching the specific politics and musk adoration that he wants it to?
I avoid Grok for meaningful token spend on purpose/boycotting. I do check in via openrouter occasionally to check it's chat performance which has seemed fine to me since 4. My total grok spend has been ~$2. I disagree with his politics to a huge degree.
My token spend at api rates is about $3000 usd a month recently.
i read an independent study that found other ai were all left of center (how ever one measures that, sentiment analysis normalized to a given population??). they said grok was evenly left/right split
but what's to validate any given population as centrist anyway
they suggested the ai opinion drift was caused by internet demographics not directly reflecting actual population i.e. California publishes more etc
Man, I got ready to debunk this but just questioned my life choices instead.
Why the fuck am I glued to my smartphone arguing with people posting shit like this instead of spending time with my daughters, creating new things or looking after my own body and soul.
Michael, please copy your comment verbatim into any LLM with search capabilities and ask for primary sources that confirm or deny your statements, and break down the beliefs by party.
Then, use your real flesh and meat brain to ponder where your (not other peoples' - your own) beliefs about this came from, and what their motivations for blasting them at you could be.
Goodbye HN, I think I'm done with this site for good.
Instead of feeding the whole thing to chatgpt let's pick one item.
We tend to accept young earth creationism because perceptively it has little to do with everyday life doesn't hurt anyone and people are quite open about it. It also means that your brain is literally broken, you have no critical thinking skills and you are just the right sort to be manipulated by evil and end up building or guarding the concentration camps.
According to Gallup 95% of Republicans believe this at a strong majority 58% believe that the earth was quite literally created as is less than 10k years.
While this is magnificently stupid in and of itself it's not the problem it's merely a symptom of their disease.
They largely believe that dead kids aren't a reason to reign in guns, they believe climate change is a hoax, a cycle, or inevitable and won't do anything about it or let anyone else. They believe that us aid to poor nations was 10-15% of our budget and we just can't afford to not let kids starve when it was one-half of 1%. They believe that it may be necessary to use violence against the rest of us to maintain the status quo. They believe that we need an stong authorian to lead us democracy be damned, they believed that COVID was a scam until the corpses piled up or even after!
Here's a hilarious one. It's a graph of the percentage of the population which understood that COVID was worse than seasonal flu a march through April 2020. For Dems it goes from 74 to 87%. For Republicans it goes from 42 to 40 it actually drops whilst body count goes up.
They are magnificently staggering wrong on average about everything. We could go point by point and I could find polls for every single one because I'm basing my understanding of their position by reading their own words and polls by reputable sources like pew and Gallup.
The right Wing collectively has departed so entirely from reality that educated con artists must carefully contort their words to avoid their insanities and prejudice whilst fools spout forth with abandon uncaring.
I certainly have some political disagreements with Musk, but more than that I would say the way he runs his companies makes me extremely anxious. The man is just always talking about stuff that never actually happens. In practice it does seem like cooler heads prevail and Grok et al. have trajectories pretty in line with other major providers...but because they're pretty in line why take the risk? Why build on foundations that ostensibly could be re-tasked to produce a "woke free" Odyssey?
Can anyone explain to me why Q and K are both needed? They only ever appear as a pair, so why can’t you just define a matrix A = QK and learn that directly?
Because the size of the attention matrix depends on the number of tokens (this is what makes attention N^2).
If you don't care about having a flexible number of input tokens (e.g. in image processing) you can learn a fixed routing matrix. This is known as an MLP mixer https://arxiv.org/pdf/2105.01601 : you have one layer that processes each token in isolation ("vertical MLP") but ignores the inter-token connections, followed by a layer that combines between tokens ("horizontal MLP") that treats the internals of every token identically.
The problems aren’t always that interesting in their own right. But the quest to solve them is often what drives the invention of new techniques.
We ultimately got the Langlands program and a large amount of algebraic number theory out of attempts to solve fermat’s last theorem. A “clever” approach using techniques from the 1800s wouldn’t have been anywhere near as fruitful for mathematics as a discipline.
reply