At the heart of inference is matrix-vector multiplication. If you have many of these operations to do and only the vector part differs (which is the case when you have multiple queries), you can do matrix-matrix multiplication by stuffing the vectors into a matrix. Computing hardware is able to run the equivalent of dozens of matrix-vector multiplication operations in the same time it takes to do 1 matrix-matrix multiplication operation. This is called batching. That is the main trick.
A second trick is to implement something called speculative decoding. Inference has two phases. One is prompt processing and another is token generation. They actually work the same way using what is called a forward pass, except prompt processing can do them in parallel by switching from matrix-vector to matrix-matrix multiplication and dumping the prompt’s tokens into each forward pass in parallel. Each forward pass will create a new token, but it can be discarded unless it is from the last forward pass, as that will be the first new token generated as part of token generation. Now, you put that token into the next forward pass to get the token after it, and so on. It would be nice if all of the forward passes could be done in parallel, but you do not know the future, so you ordinarily cannot. However, if you make a draft model that is a very fast model runs in a fraction of the time and guesses the next token correctly most of the time, then you can sequentially run the forward pass for that instead N times. Now, you can take the N tokens and put it into the prompt processing routine that did N forward passes in parallel. Instead of discarding all tokens except the last one like in prompt processing, we will compare them to the input tokens. All tokens up to and including the first token that differ, that come out of the parallel forward pass are valid tokens for the output of the main model. This is guaranteed to always produce at least 1 valid token since in the worse case the first token does not match, but the output for the first token will be equal to the output of running the forward pass without having done speculative decoding. You can get a 2x to 4x performance increase from this if done right.
Now, I do not work on any of this professionally, but I am willing to guess that beyond these techniques, they have groups of machines handling queries of similar length in parallel (since doing a batch where 1 query is much longer than the others is inefficient) and some sort of dynamic load balancing so that machines do not get stuck with a query size that is not actively being utilized.
> Less interested in people and character studies.
If you don’t want examples then all you need to know is velocity. The Y Combinator people call it doing things that don’t scale. Here is how it works for absolutely anything:
1. Get the right tools in place. This is an intrinsic capability set you have to build. People tend to fail here most frequently and hope some framework or copy/paste of a library will just do it for them. Don’t be some worthless pretender. Know your shit from experience so you can execute with confidence.
2. Build a solid foundation. This will require a lot of trial and error plus several rounds of refactoring because you need some idea of the edge cases and where you the pain points are. You will know it when you have it because it’s highly durable and requires less of everything compared to the alternatives. A solid foundation isn’t a thing you sell. It’s your baseline for doing everything else at low cost.
3. Create tests. These should be in writing but they don’t have to be. You need a list of known successes and failures ready to apply at everything new. There are a lot of whiners that are quick to cry about how something can’t be done. Fuck those guys and instead try it to know exactly what more it takes to get done.
4. Finally, measure things. It is absolutely astonishing that most people cannot do this at all. It looks amazing when you see it done well and this is ultimately what separates the adults from the children. This is where velocity comes from because you will know exactly how much faster you are compared to where you were. If you aren’t intimately aware of your performance in numbers from a variety of perspectives you aren’t more special than anyone else.
People who accomplish hard things are capable of doing those because they didn’t get stuck. They had the proper tools in place to manipulate their environment, redefine execution (foundation), objectively determine what works without guessing, and then know how much to tweak it moving forward.
I completed the second edition years ago and found it illuminating. It's very well done! The explanations pertaining to OS dev are nice but it was also a useful artifact for learning more Rust after having gone through the Rust book. The author had a good grasp of the idiom of the language, so seeing how he expressed certain things in code vis a vis how I would have done was educational. The blog's explanation of how futures are implemented in Rust was especially a highlight[0], and I think it's the best I've seen on that topic anywhere.
I hope Phil will find the inspiration to come back and work on the EFI-based third edition again. AFAICT, (based on commits to the git branches) that has stalled. I was a GitHub sponsor for about a year until my CC expired and was always bummed to see it left out of the "This Month In Rust OSDev" posts.
Anyway, the second edition is still a great way to spend a weekend's dev time and I appreciate all the hard work that clearly went into it. Thanks Phil!
I've travelled to Vietnam a lot over the past couple years for work and my SO's family, and there is a lot of rot in the VN economy that has finally caught up to it.
Most of the growth could be attributed to Korean, Japanese, and Chinese conglomerates spending a LOT of money moving factories from China to VN in the 2014-22 period after China started trade wars with Japan, South Korea, Taiwan, and the US.
While this lead to massive capital expenditures, the average Vietnamese person's life didn't change too drastically, as most of the investments were located in Hanoi (Chinese) or HCMC (Korean+Japanese) and only captured by the capital owning class. While there are more FMCGs and brands now, almost everything has a luxury tax on it or is imported from the US, Thailand, or South Korea, and even Temu quality clothes cost as much as they would in the US, and furthermore - rent is insanely high in HCMC and Hanoi, so 50-60% of your income is spent on rent alone.
Furthermore, party apparatchiks decided to maximize the graft they could partake in by investing heavily in boondoggle hotel and apartment projects (similar to the Ghost cities you see in China and India).
When the COVID pandemic happened, the gravy train ended.
Vietnam followed a zero-covid policy that caused migrant workers to return to their hometowns, and there was a massive corruption scandal around COVID testing infrastructure and repatriating Vietnamese migrant workers from abroad.
This was also around the time an internal conflict began brewing between the Northern VCP (supported by the VinGroup billionaire who is also from Vietnamese Military royalty), Southern VCP (supported by this billionaire), the Army (they own Vietnam's largest telco and most of the PSUs), and the Ministry of Public Security (they have the dirt on everybody and are the actual enforcers in Vietnam).
Basically, the MPS backed the Northern faction and won this round, and the Army stayed out of this fight in order to keep making money.
That said, all this upper level turmoil (under the guise of a corruption purge) dented investor confidence recently, along with some very prominent infrastructure collapses like the power outages and the internet slowdowns.
For example, this past Tet people didn't spend as much on fireworks or red money compared to the 2023 Tet, real estate projects have started slowing down, and it's not uncommon to see abandoned construction projects once you leave HCMC or Hanoi.
If you're part of the expat Bui Vien/D1/Thao Dien scene you wouldn't notice this kind of stuff (they're too busy huffing balloons/nitrous despite it being banned in VN leaving surgeons to physically restrain patients), but once you go to "real" Saigon (eg. D10, D8) let alone outside of HCMC/Hanoi/DaNang/DaLat these issues become very prominent.
I can do the same with almost any IP camera and even USB one with low latency and maximum quality the camera can supports. If anyone is interested in such setup I can do a quick write up about or maybe a repo with the configs needed.
I personally prefer Bosch and Hanwha cameras. Great optics, low light performance and solid firmware. Axis tends to be expensive and low light performance is not as good, for the price. UniFi Protect cameras are decent, but the standalone firmware is rather limited.
I connect these cameras to Frigate[1] locally.
Some example model numbers:
Bosch Flexidome IP Starlight 6000 Dome Security Camera - NIN63023A3; Hanwha Techwin XNO-8080R WiseNet X Series Network Bullet Camera 5MP 3.7-9.4mm; AXIS P1468-LE Bullet Camera
The crawler doesn't interact directly with pagerank. PageRank is computed later and is attached as a per-document value.
I asked Jeff about what it was like in the early days and he told me: when they first joined, the entire crawl to index to serving stack was documented in a README that you would follow, typing commands and waiting for each step to complete. A failure in a step meant completely starting over (for that step) or even earlier, depending on how and where temp data was materialized.
He said he and Sanjay (Ghemawat, not Gupta) then wrote mapreduce as a general purpose tool for solving multiple steps in crawl to servable index. Not only is mapreduce good at restarting (if the map output and the shuffle output are persistent), the design of mapreduce naturally lends itself to building an indexing system.
If you go back to the old papers you'll see several technologies mentioned over and over. protocol buffers, recordio, and sstable: the first is an archive format to store large amounts of documents in small number of sharded files, the second is a key-sorted version of the same data (or some transformed version of the data). So, building an inverted index is trivial: your mapper is passed documents and emits key/value pairs (token, document) if a token is in the document. The shuffler automatically handles grouping all the keys, and sorting the values, which produces a fairly well-organized associative table (in the format of sstables).
BigTable came about because managing lots of sstables mutably became challenging.
|
MapReduce was replaced with Flume, which was far more general and easier to work with, and BigTable was replaced with Spanner (ditto), and GFS replaced with Colossus, but many of the underlying aspects of how things are done at Google in prod are based on what Jeff, Sanjay, and a few others did a long time ago.
Note that mapreduce isn't particularly innovative except the scaling aspects were fairly esoteric at the time.
Are there any servers and firmware that I can use to replace the software on my Roomba?
I spent nearly $1k on my Roomba. As a single person with a dog, it helps me a lot. My allergies aren't nearly as bad, I don't have to vacuum by hand nearly as much. I liked that with the map the bot was a lot smarter and didn't bump into or damage my things.
I feel like by selling to a company like Amazon that the rug is being pulled out from beneath me. I am truly disappointed in this move and all the people involved.
Having been on both sides here are my 2c. Pardon any brevity since i’m on mobile and in transit. Id advise reading up on these at this excellent guide (1).
Comp at startups are generally a sliding scale of cash to equity. A typical offer will ask you to slide it one way or another.
To judge stock, ask for,
- 409a valuation to know the current strike price of the shares
- total outstanding fully diluted shares to know the total shares available
- size of the employee option pool (eg, 10-15pct)
- possibility of an 83(b) election
- ISO vs NSO - what kind uf options are they?
- re ups, and anti dilution clauses?
- triggering events (eg what happens when the company gets bought?)
The more you know the better you judge the value. If folks are cagey in giving details, definitely push back and ask why.
Next, consider that the company’s progress is all that determines your shares net worth.
- how much do you believe in this team, space and product?
- and ask yourself - are you able to completely push this into a “lost cause financially” bucket in 5y? or do you need the cash? i’d advise being comfortable with the former :)
lastly, look at how much value you bring to the table to determine how much you get. if you’re engineer 1 with two non tech cofounders - you’d be worth much more than engineer 10. In that case, look at some of the blog posts online (esp by folks like Leo Polovets) on some ways to think about these numbers.
Udemy has a few good tutorials, but on Youtube check out Tutemic's code architecture course[0], HeartBeast[1], GDQuest[2], GameFromScratch's Godot tutorial series[3] and CodeWithTom[4].
And of course on the blue website /r/godot and /r/madewithgodot
Some notes from experience from the other side of a failed chemo:
- Not all ports are metal. Some are plastic.
- Not all chemo drugs have the same side effects; even within drugs, not all implementations have the same ingredients.
- Lots of chemo retrospectives talk about "nausea", but that's imprecise. Nausea and vomiting are separate things, and can be invoked separately. Some chemo drugs act on the vestibular system, which is the part of the nervous system that decides whether the body needs to vomit; some chemo drugs in your blood basically yell "PUKE PUKE PUKE" at your brain, and you vomit without nausea. Other chemo drugs do a number on dopamine or serotonin receptors, giving you intense nausea without the trigger to puke (but you might anyway). You can tell if you're in the latter bucket if chemo makes you absolutely hate chocolate, turkey, or other tryptophan- and tryamine-heavy foods — without the serotonin/dopamine boost in most chocolate, it's just a weird bitter taint that ruins sweet flavors. (Also, some chemo drugs make you REALLY crave fructose. By the bucket. Fruits, fruit-flavored candies, fruit juice, non-stop.)
- If you're young and relatively healthy, there's a decent chance you'll deal with your port closing up, especially if your cycles are 3+ weeks in length. Flushing the port is time-consuming; chemo is already hard to schedule around, but if you have port issues, nothing is ever going to take just 2-4 hours.
- Not sure if Charlie had to deal with pre-meds and post-med flush drips, but we did, and they're mandatory — even if you're unable to finish the infusion.
- Some chemo drugs have a slim chance of causing a severe reaction. If you're going to have one, it'll be within the first 10 minutes of the first infusion. If you have one, it might be a classic sudden allergic reaction, in our case Taxol caused the throat to close up, losing the ability to speak or breathe, treated with a heavy dose of Benadryl. Regardless of the specific reaction, if it's severe then it's a massive problem going forward, because your options are to either ditch the oncologist's first-choice drug or (with a ton of suffering) try to build up a resistance in a hurry. And yeah, if you're unlucky enough to have that reaction, you're very likely going to have that violent reaction in front of an audience of everyone else getting an infusion.
- One of the side-effects of chemo isn't listed here, and might not be a factor in his combo, but in general chemo has an eventual side effect point where it can't continue: the potential to cause nerve damage. This will look or feel a little like Parkinson's, with numb extremities, loss of motor functions, difficulty stringing together words/actions. At the first, slightest sign of this, chemo is over, sometimes for good. If the cancer isn't getting treated by then... good luck.
- Holding off on a cycle due to low white blood cell counts is a foreign concept to me. They ran tests and infusions between cycles to keep WBC counts up, but any cycle delays were considered suicidal. Again, might've been just our treatment course (and severity; we started at stage 4).
- This will vary between infusion centers, but as long as COVID is a problem, you can't rely on having in-person company with you, or contact with other people getting infusions (on account of everyone involved having massive immunity problems from the chemo). If you crave contact, bring a phone, a charger, and the longest possible cable you can get for that charger.
- Nobody cared to hack the PS3 because it had Linux. There was an air of mythos about the security being top notch, but nobody skilled tried. Linux was restricted to not having 3D accel, and there was a short-lived Linux side exploit that granted you that capability, but Sony patched it as expected.
- Sony released the PS3 slim without OtherOS. They claimed they didn't have the resources/interest in developing Linux for it (this was later proven to be bullshit, the patches required were trivial).
- Some folks started staring a bit more seriously at PS3 security at this point.
- geohot developed a hilariously unreliable physical RAM glitching attack to escalate from OtherOS to the hypervisor on the Fat (note: this does not grant access to the secure SPE where all the crypto/DRM is done). I guess he was aiming for the slim, but this was his first target.
- Sony got scared and illegally pulled OtherOS on the Fat in an update, violating consumer protection law (they got sued for this and lost)
- Everyone started looking into PS3 security at this point.
- "Someone" (I have my suspicions) developed and sold a USB dongle which exploited GameOS to gain access at that level, and which included code directly and solely intended for game piracy in its payload. This only broke GameOS security (not hypervisor nor secure element), but Sony's actually-terrible system design meant that was enough to pirate games.
- I and others reverse engineered that and I wrote a Linux loader (AsbestOS) that could be used with clone exploit devices, and worked on both the PS3 Fat and Slim. These payloads didn't include any piracy-related code, just a Linux loader. This is Linux running in GameOS mode, which gives you 3D access (though no drivers were ever written...) and required some kernel patches due to VM configuration differences. I worked on those patches and they are now upstream.
- Research continued, and I and others at fail0verflow eventually discovered that Sony had completely borked their crypto and we could calculate their private signing keys. Due to a technicality, doing that calculation required having access to the symmetric crypto key beforehand, which meant that for any given secureboot chain level, you first needed a (one-off) exploit at that level to dump the AES key, then you could calculate the ECDSA signing key.
- We gave a talk detailing both an exploit at a certain level and the crypto flaw, without disclosing any keys.
- A week or two later geohot used an exploit he had stashed at the metldr level (that's the second highest level, after the bootldr cold boot code) and our method to calculate and publish the keys for that level on his blog. Initially without crediting us (this is a common geohot theme) until I asked him to. metldr cannot be updated, so this is close to a complete hack (Sony found a cute platform recovery trick later, wrapping things with bootldr, until finally someone exploited that and it was all over)
- Sony sued geohot and me and a bunch of others. They probably thought we were cooperating or something. That was a stressful few months.
- geohot settled with them and promised to never hack their stuff again, and they dropped the other defendants (myself included) from the lawsuit, but not before causing personal and legal trouble to some of us.
Most interviews do not have enough time to assess for culture fit (from the candidate), so ask for time. Any company that has a good culture will be happy to give you an hour to ask questions, maybe more. This might be only at the offer stage, but by that point they are invested enough that there's no reason other than hiding a bad workplace to not give you that time.
Treat it almost like an interview. Write up questions, think about the questions for a while, think about what a good and bad answer looks like. They're probably going to be pretty open ended, and the conversation may be somewhat circuitous, but they will be valuable.
Questions I like to ask are things like:
- Walk me through the process from ideation, spec, code, testing, deployment, maintenance. Who owns each part? How long does each part take? Which people/roles, processes, tools/tech are involved?
- If you left, what would you want to take from this team to your next? What does the team do better than others?
- What do you think the team needs to improve at? What does it do less well than other teams you've been a part of?
- How does change happen? What mechanisms do you have for change? Can you give some examples of change happening?
- How does the team feel about... Code Review (or anything else you care about)
No income tax on business or personal income. You can become the state’s resident after spending just one night in a hotel and opening a bank account. That’s partially why it’s highly favored by RVers who choose to longer have a permanent home.
There’s opportunity to modify the state’s politics. ;)
Sony's image sensor fab business is a bit different.
First of all, there's two companies: Semiconductor and Imaging. Sony opens some of their technologies to other customers by transferring Imaging unit's patents to Semi and allows them for re-licensing. So, not everyone is using the same sensor even if they are coming out from the same fab.
Second, there's customer tailored sensor business. Sony can provide a customer a baseline to start and, customer can customize this sensor according to their needs and got it produced for themselves.
Lastly, there's strict division between customers in semiconductor side. So Sony Imaging can't access to specs and design of other customers' sensors.
How it works in real life?
- Sony A7 series use Sony's sensors with all their secrets at their disposal but, not with Nikon's and Fuji's secret sauces.
- Nikon orders customized, "Designed by Nikon" sensors to Sony Semi. Sony Imaging cannot access them (unless they buy and disassemble a Nikon Z).
- Same for Fuji. They have less rolling shutter in XT-4 for example, and it's again built by Sony, for Fuji. No, Sony Imaging can't play with them.
- Canon R5's sensor is "Designed by Canon". I'm not sure they are producing it. Sony or Tower Jazz maybe producing it. I don't know.
At the end of the day, the fab is almost one but, designs are many and diversified and, nobody can see each other's design. Like TSMC and Global Foundries.
Shameless plug: I started a YouTube channel on this topic that covers a lot of commission free trading API's, including Alpaca (got featured in their docs), Robinhood Private API, TD Ameritrade, Tradier, and more. I'm seeing a huge growth in interest in developing automated trading bots and algos. With commissions dropping to zero, there is a huge wave of day/swing traders that want to learn Python, so am creating as many tutorials as possible.
There are a number of blog posts by people such as Tim Bray, Steve Yegge, and others - as well as internal engineering blogs (at least there is one at Facebook and several at Google). I read these on a regular basis (Yegge's hasn't been updated in a while), but only to take away ideas and learn about new technologies.
Their approach is almost certainly not the right approach for you, because you are not Google ( https://blog.bradfieldcs.com/you-are-not-google-84912cf44afb ). That said, if you are looking for a position at these places, then the blogs might help you, in a number of ways. If you want to work there though, focus on the sound engineering techniques applicable anywhere (maps, hashing, trees, graphs, algorithms) and build a reputation through Open Source contributions to the projects the companies are involved with.
Read the curriculum of an undergraduate computer science course and read up on the things you haven't heard of. Some courses will even have lecture notes available.
E.g. these four pages are the university of Cambridge masters in computer science:
> Sorry, this is just rubbish. None of that money gets distributed to retirements, even if they work at Disney. What sort of lies is this?
Disney is an S&P 500 company. Many mutual funds and retirement funds track the S&P 500 and various other indices. They do this by investing money in the public companies that make up those funds.
Of course it's not just S&P 500 companies that get invested in by institutional investors. Retirement funds typically seek to generate higher returns on retirement funds from younger employees. To do this, they have to invest in riskier businesses, which typically means smaller investments in larger numbers of newer businesses. Thus, to simplify things, we can pretty much say that any public company traded on a major stock exchange can (and does) affect the price of retirement savings accounts.
At the core of it, every public company, regardless of their business, is simply an investment vehicle in which people can put wealth in the hopes of competing successfully against inflation. The creation of a share price increase greater than inflation increases the wealth of the shareholders, and creates new money from thin air.
This creation of money is how all people can afford retirement. Simply saving your money in an account which keeps the nominal amount of cash would lead to loss from inflation. Most people will not contribute enough money to have enough in retirement savings. Thus, they depend in some way on the creation of money by private enterprise to subsidise their retirement.
This is true regardless of whether retirement accounts are self-directed (as in a 401k), centrally managed (as in a pension), or government run (as in social security). The connection to self-directed and managed accounts is obvious. Governments of course rely on the creation of money to fund tax revenue that is then put towards government retirement programs.
More alternatives used in markets in Shenzhen China
www.icgoo.net
www.ickey.cn
www.hqew.com
www.szlcsc.com
And for more international parts (still used in mainland a lot)
www.eciaauthorized.com
Octopart is a more generic site for discovering chips and getting an idea of the price, but still it fetches crappy prices from Digikey, Farnell, Mouser, etc.
If your product is being made in China and you are making the design choices, a quick Taobao search (or using the above links) is great to see how popular a part is. Not recommending to buy parts from Taobao though (unless you are more seasoned). There are also alot of agents that can source parts maybe 1-2 levels below the original factory. Their prices are so good it's unbelievable but they are hard to find and validate. Having volume also helps.
I learnt a lot more than I ever wanted to know about every aspect of elevators from Elevator Hacking: From the Pit to the Penthouse with Deviant Ollam (who seems to be like a professional elevator pen tester) & Howard Payne, from DEFCON 22. Highly recommended. You'll be talking elevatorese like a pro after you watch that.
As someone who has recently started figuring out this whole ADHD thing, definitely seek out help for it. There are a ton of really powerful tools and support systems that make a tremendous difference. Especially organizational systems designed for people with executive function deficits (whether that's ADHD or not, doesn't really matter!)
And additionally, if you find that these things speak to you, medication makes a tremendous difference. I really can't overstate the degree to which it can change your life.
My personal passion is the "desert blues" music that's mostly Tuareg. Start there with Tinariwen and Bombino. They're widely available in the West without much effort. They also tour heavily in the US, so keep your eye out for concerts!
From there, the Sahel Sounds label puts out a lot of obscure and interesting music on Bandcamp.
I can recommend two films as well. The first is "Akounak Tedalat Taha Tazoughai" (Rain the Color of Blue with a Little Red in It), which is a Tuareg remake of Purple Rain, the first fiction movie in the Tamashek language. The second is the documentary "They Will Have to Kill Us First", which is about Malian musicians living in refugee camps due to the civil war in northern Mali.
That documentary leads to the band Songhoy Blues, who are from the same area but Songhoy rather than Tuareg. They are young guys who met in a refugee camp, and managed to score a British record contract. Upon seeing them live, my spouse said it was like seeing a young Rolling Stones. Very accessible, compared to more raw music like Tinariwen.
1. Precalculus: Precalculus: A Prelude to Calculus - Axler
2. Calculus: The Calculus Tutoring Book - Ash.
College:
3. Preparation for Collegel-level maths:
3a. General prep for high level maths: How to Study as a Mathematics Major - Alcock
3b. Proof writing: How to Prove It - A Structured Approach - Velleman OR Book of Proof (2nd ed) - Hammack (it's free!)
4. Mathematical Analysis:
4a. Good prep for Analysis: How to Think About Analysis - Alcock
4b. Understanding Analysis (2nd ed) - Abbott OR Yet Another Introduction to Analysis - Bryant (has full solutions) OR The How and Why of One Variable Calculus - Sasane OR Mathematical Analysis - A Straightforward Approach (2nd ed) - Binmore (has full solutions)
5. Discrete Mathematics (a combination of set theory, combinatorics, a bit of discrete probability and graph theory): Discrete Mathematics - Chetwynd, Diggle
6. Linear Algebra: Linear Algebra - A Modern Introduction (4th ed) - Poole
The presenter speaks slowly (as the name suggests) and clearly, and there are over 100 episodes on a variety of topics, so there's probably something to suit your interests and level.
Deutsche Welle also have podcasts and videos at various levels and you can download a slowly-read news broadcast ("Langsam gesprochene Nachrichten") every day: http://www.dw.com/en/learn-german/s-2469
Or for a normal speed news broadcast for native speakers, you can subscribe to ZDF's "Tagesschau" or "Tagesthemen" podcast, or watch a video of "news in 100 seconds": https://www.tagesschau.de
https://www.kernel.org/doc/ols/2007/ols2007v1-pages-225-230.... http://www.haifux.org/lectures/312/High-Level%20Introduction... https://zserge.com/posts/kvm/