Adding to the prior comments as my intuition matched yours, there’s a nice Reddit thread that gives some context into how it can be faster even if you require exact matches: https://www.reddit.com/r/LocalLLaMA/s/ARxHLqRjdM
The TLDR/key (from my understanding) is that verifying N tokens can be faster than generating N tokens.
> The TLDR/key (from my understanding) is that verifying N tokens can be faster than generating N tokens.
Yes. This is because to generate token n+1 you need token n etc. So generating from scratch is a sequential (thus slow) process.
When we verify tokens, we can, for each token, use all preceding tokens as input and check that the output token matches the expectation. But since the full sequence we want to verify already exist, we can do it in parallel for each token we want to verify and not sequentially.
This is why training transformer models is much faster than RNN, we do the same thing during training, it's just that the sequence we compare to is the ground truth and not coming from another model.
Congratulations to their team. SQLGlot is a really powerful tool that a lot of companies use so a huge contribution to the OSS community so hopefully it continues to be supported and gets better and better!
Feels like two types of companies raised money: - Companies trying to couple the cloud with a programming language. - More recently, companies trying to couple GPUs with a programming language/alternative to CUDA.
If you are running things locally (I would think especially on the edge, whether on not the LLM is local or in the cloud) this would matter. Or if you are running some sort of agent orchestration where the output of LLMs is streaming it could possibly matter?
Uber's sales reps convinced my company to sign up for a business account for employee travel. Soon we started noticing that advertised fares for the exact same ride were 20-40% higher when the business profile was activated. Now the guidance for employees is to always book rides using the personal profile and file for reimbursement. I'm convinced that Uber is only able to survive by scamming customers.
FWIW I used a business profile a few years ago because that's the only way to not use your uber gift balance (we couldn't expense gift balances), and didn't notice any discrepancy between personal and business rates for the base fare. Selecting "business" disables any promos, which is likely where the discrepancy is coming from.
The deals for any of these meal services are bullshit in my experience. There's a lot of hidden fees or minimum order amount stuff that isn't apparent until you get to the checkout, definitely not the great deal they have you believe you're getting.
We have a restaurant where I can order a $10 meal and pick it up myself. The same meal done through uber is $30. Everything has a percentage added and there's at least 3 extra "you used us" fees that get tacked on. All the menu items can have anywhere from a 20->100% markup. It's quite insane.
Today there are so many other solutions: Stytch, Descope, PropelAuth (For B2B companies), and others.
VCs went a bit ham on this category when Auth0 got bought. I sense that the general thought process was: Auth0 multi-billion dollar company -> Auth0 will become crappy under Okta > replace Auth0 and build a bit company.
Basically people are constantly calculating metrics based on existing tables. Think something as simple as a moving average or the sum of two separate columns in a table. Once upon a time you would set up a cronjob and populate these every day as a SQL query in some python or Perl script.
Dbt introduced a language for managing these “metrics” at scale including the ability to use variables and more complex templates (Jinja.)
More broadly dbt did two other things: 1. It pushed the paradigm from ETL to ELT (so stick all the data in your warehouse and then transform it rather than transform it at extraction time.) 2. It created the concept of an “analytics engineer” (previously know as guy who knows SQL or business analyst.)
Interestingly, according to Axios, the price was pretty limited: "Amplitude (Nasdaq: AMPL) acquired CommandAI, an SF-based software user experience startup, for $20m (net of cash). CommandAI (fka CommandBar) had raised around $23m from Insight Partners, Itai Tsiddon, Thrive Capital, and BoxGroup."
I would be curious to learn more about the rationale to sell the business as I understand the strategic value to Amplitude. Interestingly, these next-generation digital adoption platforms have generally been pretty challenged.
The number that was reported in term sheet / pro rata did not include any stock, which was a big part of this deal. The total consideration for CommandAI was north of $45M. We're working with them to issue a correction. Long story how the wrong number got reported- mostly too much telephone. It was also a good outcome for their investors in that they were able to return more than what was raised.
We at Amplitude approached the Command AI founders about joining Amplitude. They had a bunch of runway and good growth numbers. They were initially planning to raise another round of fundraising to continue to scale their business instead. It took some time on both sides to figure out what the win-win combination would look like.
We're honored that they decided to combine with Amplitude and are excited about what the combined products look like.
Command AI co-founder here. What I'll add to Spenser's nice comment is that there was this strange tension when we were deciding to do this between: (1) Feeling like the only right outcome for a growing startup with runway is to keep going and try to build a massive independent company, (2) Realizing that we can probably grow faster as part of Amplitude (obv with less upside capture)
Obviously if your growth is so epic and you're a top 1% company, the choice is obvious. But there are a lot of companies that are doing well outside of that group that I don't think allow themselves to consider the acquisition route. We were like that until Amplitude reached out and we got good counsel to seriously consider their offer.
You almost don't have to snoop for details to have a good guess at what happened. If the acquisition announcement is the first time in a year or more that you're hearing from this company and they omit the sales price, you can bet this isn't the exit most parties wished for. Although by this stage the founder(s) is probably happy to find a buyer who will retain their team.
What’s interesting is how much it contrasts with TechCrunch’s story: ‘Most of Command AI’s 30-person, San Francisco-based team will be joining Amplitude. Command AI’s co-founder and CEO James Evans wouldn’t reveal the terms of the deal, but said candidly that an acquisition wasn’t something he’d been planning on.
“Our growth was great and we had plenty of runway,” Evans told TechCrunch. “We weren’t out shopping ourselves or anything. But when Amplitude reached out a little while ago — this summer — we got really excited about the combination and became convinced that we could grow faster and reach more users together.”’
People want to save face. Some more than others. I also read the TC article and thought they were trying a bit too hard to make this seem like a good outcome.
Nothing against the founder. It's just how the game is played. And there's little to gain from deviating from the norms.
Edit: It benefits not just the founder’s ego but also the future career prospects of the employees. Big difference in your engineers being able to say “I worked at X all the way until they got acquired!!” and “I worked at X but the product was so unsuccessful we had to have a fire sale.”
That's too bad that the TC article read that way. They returned more money to investors than what was put in, which puts them in the top ~20% for acquisitions of this scale. The crazy part is even a lot of the 1B+ privates aren't able to do that (eg Lacework)
> you can bet this isn't the exit most parties wished for. Although by this stage the founder(s) is probably happy to find a buyer who will retain their team.
I've seen a startup that was bootstrapped 10 years ago but took on too much money and ended up getting acquired for "undisclosed sum" but at the same time everybody knew the investors were just recouping their money while founders got nothing (they had sold their equity for raising more money as their costs were way up but nobody was buying their product). Cue local news reporters don't know the nuances so they'll just announce "Startup X acquired by Y, wow!"
Founders place a lot of emphasis on getting "acquired" but there is roughly 95% chance of you selling at breakeven where the terms leave you with a year's worth of salary as a junior SWE
Get ready to see a lot more of these "acquired" news coming out in the near future. something around less than 1% chance of getting more money than you put in.
Startup has horrible odds especially if you are building a simple GPT wrappers
Been watching this episode unfold on Twitter and has read about Matt’s domain hijacking of thesis, etc.
It seems to me like Matt is the type of person who likes to hide behind character and other ad hominem attacks rather than addressing actual issues at hand. Perhaps this is because of a psychological issue but I can’t really know. Normally I would think the community would be repulsed by this and would find an alternative.
What is remarkable is that there has not been enough community animus to fork relative to Terraform and OpenTofu. It shows the power of the underlying GPL license and “plugin” approach. Something to think about for other companies that may be thinking about relicensing versus just building a durable ecosystem around their proprietary brand and assets.
The TLDR/key (from my understanding) is that verifying N tokens can be faster than generating N tokens.