If you cannot afford doing it, then you cannot do it. What's the purpose of having the capability to do something if you cannot afford the losses of doing it?
Not in many tasks. I use deepseek as a fallback in https://phrasing.app and it’s always very apparent when it happen (due to mistakes/clear performance drop off)
I feel like there's an implication here that distillation is a problem but I don't understand what you mean. I thought distillation was generating text from a model and then training another model on it. Is the something unethical in that? You're paying the API costs to generate the tokens, right?
Or I guess more to the point: is this something frontier labs have said is (or tried to paint at any rate) problematic? This feels like an "out of the loop" situation because I've only ever heard "distillation" with a positive connotation before.
That OpenAI was in the wrong when they ignored everyone copyright, does not make it right to ignore their ToU. If a one wants IP and rule of law (incl contracts) to be respected, one should not violate others rights when it is convenient.
On a more risk-strategy level there is the size of their legal team, general endowment, and supplier and political connections to consider.
Everyone is free to ignore their ToU, but I can understand why a company would avoid it...
it doesn't matter the reason. This is a race and nobody will care or remember how the winners got there.
Mistral looks like it's fading away to irrelevance unless they can play alongside the similar sized models, or have some unique advantage other than being in Europe, for Europe. I was really excited for them back when they were startup that had the biggest European venture round ever. This space will have a few winners, and many losers. Google, plus either Anthropic or OpenAI most likely. Big models will see breakthroughs in inference performance/cost fall precipitously and small models will only exist on devices (Pixels and iPhones, cars, watches, bluetooth speakers, etc)
It’s not that I don’t agree with you, I am just pointing out why it’s hard to catch up to scaling laws given the European economic (capital) and political (US would be upset if they found out Europeans distill) constraints. China is only bound by economic constraints.
> This is a race and nobody will care or remember how the winners got there.
For consumer AI, yes. For coding assistants, probably.
For specific application "business" AI like the things Airbus announced the other day? Not at all. What matters for an Airbus using Mistral to build compliance documentation based on AI generated physics simulations is the enterprise relationship, reliability, compliance, forward deployed engineers helping with the fine tuning, quality, predictability, support. A Chinese lab having a better at benchmarks model that is cheaper is just irrelevant for that.
And IMO, the real money in AI is this type of "business AI" deployment. Developer tooling tends to converge on becoming commoditised. Once you're a core supplier for a big bank and embedded in their processes, you're there untill you screw up with the pricing (like Broadcom), and even then.
So then the European ones should join with European copyright holders to sue OpenAI/Anthropic and watch them trying to BS their way around what they train on.
Training a model on copyrighted material is fair use and copyright holders already operate a mafia-style extortion ring in Europe, so I don't think that's a good idea.
It’s really a pity, why can’t they feel superior while breaking ToS and copyrights just like Americans can feel superior over Deepseek while breaking ToS and copyrights?
I always laugh about those ridiculously large water bottles American carry and how they remind you all the time that you must drink water as if I did need it. I wonder why that happens.
I chat to it in English instead of my native Spanish not only because of performance, but because I cannot stand the unnatural style it has in Spanish.
What do you find yourself gravitating to? What part of your job comes easiest? That things are easy to you that other’s find difficult? What do you spend time learning more about even when you don’t have to? Those are directional. For me the first time I started writing code I knew that’s what I’d need to do for a living.
Thanks for your comment. I spent the last 5 years doing a PhD in NLP that I defend next month, which is a solitary endeavor. I don't feel particularly good at it, especially given the competitive moment of the field. I am trying to transition to industry but landing a job it's being hard. I think that I am good at being a link between people and maintaining a good atmosphere. I realized that I liked people more than I thought before and that working full time in front of a screen is something I would rather not do the rest of my life. On the contrary, I feel gravitating toward social sciences, economy, culture, governance and stuff that is not very close to my professional/studies expertise.
Like with many things, finding the best is hard, finding a good enough fit is a lot more attainable and maybe should be the goal.
If you work on projects in groups often, you might be able to find what fits you by what things you end doing especially if you do those parts well. Do you read and interpret the directions, do you do the assembly, do you keep the group on task, do you verify the output is acceptable, do you figure out how to proceed when there's a problem, etc.
Also, what tasks do people who know you ask you to help with; especially if those people have choices for who to ask and then specifically ask you. Those are things that likely fit you; especially if you get enjoyment out of doing those tasks, beyond the enjoyment you might get from doing any task for someone. Sometimes, you might get asked to do these things for reasons other than you're good at them, or you may be good at them and also hate doing it, etc; so like be aware of that.
If you're lucky, what fits you is distinctive and commercially apprechiated. But not everyone has those fits, so it's good to also look for things that fit well enough to pay the bills. You may need to develop other skills to get into a position to use your good fit as well.
Thanks for your comment. I spent the last 5 years doing a PhD in NLP that I defend next month, which is a solitary endeavor. I don't feel particularly good at it, especially given the competitive moment of the field. I am trying to transition to industry but landing a job it's being hard. I think that I am good at being a link between people and maintaining a good atmosphere. I realized that I liked people more than I thought before and that working full time in front of a screen is something I would rather not do the rest of my life.
Force yourself to try a lot of things. Look for meetups in different topics you can do in your free time (I hope you have some). Try HAM radio (you can get a career in COMMs), public speaking (executive leadership), group fitness (become a trainer), maker scene (robotics) etc. (Also give me some ideas!)
Ask people who know you well what you are talented for. Oftentimes we don't see it ourselves. As you get good at something, it become easier, and you think of it as a given. On the opposite, we tend to over appreciate what is difficult for us.
Are you an extrovert or introvert? Look at how you spend your time. Do you have to spend time with people or have to be alone sometimes?
What do you do when you have nothing else to do? I know that's really hard these days with all the distractions we have. So maybe what do you watch or read about? What are your interests?
But the world changes. I started out as an engineer and that got shipped to China. I pivoted to IT, shipped to India. Pivoted to technical writing and now there's LLMs.
I figure things out and share to make it easier for others too. That works in a lot of industries.
Due to personal and professional reasons I've spent too much time alone last years. I really noticed that I love being with people and that I cannot stand being all day in front of a computer by myself. I also read a lot about history, economy, culture and geography, but on the contrary my background is in Physics and I am finishing a PhD in NLP in which I don't feel very good.
A lot of pop-psychology doesn't hold up when subject to empirical review, but OCEAN / "Big 5" does, and it's probably a decent starting point.
E.g. if you are low in extraversion and agreeableness, you probably wouldn't make a good nurse or waiter, but you might not make a bad lawyer or engineer.
If we’re being honest, highly agreeable, extroverted, conscientious, and non-neurotic people are simply going to be better suited to all forms of employment than the inverse. But, since personality is pretty durable, it’s easier to try and find a career where your weak spots are detriments, but not crippling.
I'm highly agreeable, and I've had to learn not to be. Knowing when to challenge people - "strategic non-agreeableness" - is extremely valuable. I've also made most of my career off being somewhat neurotic - I've described the core of my job as "finding things to panic about before they happen" (I went on Prozac a while back and caused an incident in the first couple weeks during uptake because my anxiety didn't trigger about something during a deploy). As far as extroversion - friends of mine who are genuine extroverts about went crazy during the pandemic, while I and a few other introvert friends got some of our best work ever done during that period. There's a spectrum - you can't be a misanthrope, but being able to take (and stand) quiet time to focus on a problem is absolutely an asset. With regards to conscientiousness, this often manifests in the workplace as an unwillingness to deviate from the plan when circumstances demand it and a preference for adding process as a kind of panacea for any kind of failure or delay, and at risk of offending the more conscientious among us, I have not found that a recipe for success.
I disagree a bit with the neuroticism and agreeableness being so obvious. There are many professions I would be TERRIBLE at precisely because I am so agreeable. And, I have real world experience with a business partner that is MUCH higher in neuroticism than I, and much less agreeable. Both sides of that spectrum have their strengths. We often have opposite approaches sometimes, but both can work, and one isn’t obviously better in all circumstances.
And introversion can be a wonderful asset in some professions as well.
However, I do agree that conscientiousness is probably pretty universally better.
There is research that suggests highly agreeable people do not do as well e.g. negotiation tactics. What is probably true is that is good to 'appear' agreeable. The same research suggests you are correct about the other 3 traits.
A highly agreeable housing inspector isn't going to be better at their job than a disagreeable housing inspector. I want my housing inspector to be harsh, unforgiving, and not grant the benefit of the doubt.
A highly extroverted person isn't going to make for a better overnight custodial worker than someone who prefers a more solitary lifestyle.
An actor who can tap into the emotional currents of high neuroticism in their work can offer a more sincere and authentic performance than an emotionally flat one.
Low conscientiousness correlates with risk taking and can be an asset in roles where over-planning to the detriment of acting can be costly - think firefighters.
I agree with this if what you mean is that employment generally requires conformity, passivity, accepting low autonomy, low creativity, etc. Otherwise, this isn't my experience.
Low agreeableness can be a positive up to a point. As a technical person, you shouldn't agree to do things that you know will not work. The technical facts have no agreeableness at all and need to be handled as such.
reply