While I think F# is a great programming language. The article missed some good uses of it.
Specifically the article missed one of largest F# deployment, in production, in the world at this point. We use F# at Jet.com and it powers every part of our core business from our dynamic pricing algorithm to search and analytics.
Over 4 million customers already on jet and over 2200 cores on azure all running F# code.
Have you ever used Jet? Try it, add a few things to your cart then add a few more and see what happens to the prices of the previous items. There are millions of permutations being computed behind the scenes.
The system is computing prices all of the time based on many factors. Plus we have built everything in house from our warehouse management system and supply chain tools to order management and everything else.
That amount of compute encompasses QA, Dev environments and any experimental and R&D work we are doing.
Plus jet.com is trying to compete with Amazon so that means the system has to be ready for many million more users than there are currently shopping with us.
I haven't used Jet, no. I'm not trying to call you out, just wondered why it was so large. I have to deal with a similar amount of pricing complexity (actually, probably more complex than Jet) in my line of work, and we don't need anywhere near that amount of resource, but granted we don't have your numbers either, and our setup is much easier to silo groups of users - so probably not directly comparable.
My comment wasn't so much about languages and frameworks (we use both C# and F#), but pricing complexity. I'm in healthcare software, and the complexity of pricing is crazy.
We have a system that joins hospitals, primary and secondary care providers together. Each hospital might have it's own prices, as will their consultants, etc. Insurers have standard price lists, but some consultants and practices have direct deals with them, which can affect the price (and the creditor for the invoices), we also support rule-sets for modifying prices based on location, clinician, day of the week, availability, benefit maxima, credits, etc. We support the notion of inherited prices, where a standard template can be setup and then used as a basis for more bespoke plans etc. All of which can be manually overridden. If additional services are added then that can affect appointment lengths and costs. If the patient self-pays they may get different prices to what they would on their insurance.
Some of our customers do occupational health where they go out and get contracts with the big city firms to provide schemes for their employees. Each deal they get is different and negotiated to the nth degree, which can include various plans/tarrifs/credits, etc. Some services the patients can book themselves online, some can only be booked by their HR manager, some as follow-ups by clinicians, etc. Also rules around what can go on invoices, how billing items are grouped, who can see what (i.e. should an HR person see what services you've had etc.)
The additional complexity comes with matching that all up with efficient scheduling. We have some customers where they have appointments that involve up to 8 separate clinicians or resources; so they need to be scheduled in and priced without leaving massive gaps in the schedules. We also support call-centres that could see 1000s of practices and can query all of them. The end result looks very much like an airline booking model, but behind it all is a mass of complexity.
The combinatorial issues are large, and have definitely caused us some headaches over the years. So I fully understand how these systems can get very resource hungry. We're not in the same league as Expedia or Jet in terms of scale (although we hold more patient records than Jet has customers, if that means anything) - and as I mentioned it's relatively easy for us to silo groups of users, so I suspect some complexities just go away for us because we can filter down the potential results quite quickly, but still 2200 sounded like a lot - it was just a gut feeling. I don't understand how a shop selling a product can have difficulty calculating a price, but just as my customers have no idea of the complexity involved in calculating prices, I almost certainly don't understand the complexities that the team at Jet have to deal with.
The way we've dealt with it is to build massive data-sets of various static combinations, so if patient X on chargeband Y wants service Z then they'll be changed N. That does mean some large data updates when someone changes their pricing and rules; but it means doing less work live - although it can still be significant. This all runs on two 32 core servers.
I know healthcare insurance pricing has become the most insane invention of bureaucracy on the planet. That is why I can pay cash out of pocket and sometimes get an 85% discount. The providers don't have to chase their money from the insurance companies - private and government included.
C# for the core app, C# and F# for satellite services. Not sure on what you're asking re: platform. If you mean hosting? Then we roll our own servers in tier 4 data-centres. If you mean OS/framework: Windows Server/.NET
All our mobile stuff is thin web-layers that consume our core APIs, so it's all js. I started looking at WebSharper and FunScript for a more strongly-typed and robust web-development experience, but they're all a bit too 'awkward'.
I keep meaning to look into Elm as it appears to be a more thoughtful approach to functional programming on the client.
I avoid JS, not because I don't like it, but there are other choices. Then again, it's much easier to find JS devs than Elm devs.
I've played with Elm, but not made anything of note with it yet. The Elixir/Phoenix crowd are getting hopped up about it lately as a good fit for them.
Yeah that stood out to me too, since when do people boast about how much resources they consume?
That's 2k customers (not concurrents) per core, which is a terribly low rate. Even if they've only been running 1 month that's 1200 seconds (20 minutes!) of CPU time per customer, and it gets rapidly worse the longer you assume they've existed...
F# has fantastic concurrency support. Great support for Asynchronous operations and Multi Threading. Being a functional first language and immutable by default means most of the micro services are stateless and this means we can take advantage of concurrency like crazy and not worry too much about race conditions. I think functional languages in general make concurrency much easier not just F#.
GC on the other hand is very aggressive with all of the immutable data structures F# creates. GC in F# is very good, though, I think without good garbage collection you have a tough time in a functional world. Microsoft is especially interested in GC performance for .NET and they have explored memory dumps to improve GC so it performs well even under load and when used with F#.
I should have been more specific in the question. I'm used to Erlang and Haskell (GHC) concurrency primitives and frameworks. To reformulate: what concurrency models can you employ in F# that don't involve manually managing threads/processes?
I work with F# and have a lot of experience with Erlang (less so with GHC and it's lightweight green threading but I've heard good things about it).
F# inherits lots of .Net's primitives which at their core are thread pool based task scheduling with the usual optimized data structures you see from this world. While this might sound primitive, F# takes it further by providing very good functional libraries which abstract this away allowing lightweight asynchronous computations to be passed around as values (the type being Async<'a>).
Alone, this is pretty nice but it wouldn't feel as natural without computation expressions which are similar to Haskell's do-notation but with some generalizations and flexibility added in. This allows asynchronous code to look and feel first class. If you're curious I'd check out https://fsharpforfunandprofit.com/posts/concurrency-async-an... or check similar links on your search engine of choice.
Now, the story isn't complete. If you're comparing things to Erlang, there are also things like MailboxProcessor<'a> and such which allow other common patterns to be easily implemented. The type checking here is a nice bonus over some other approaches to message passing languages. The big minus of course is that it's still a traditional runtime. If you really want Erlang's isolated processes, it's hard to do that well outside of a runtime like BEAM. I also think F# could use better fault-tolerance primitives but I'm actively working on this with heavy influence from my work with Erlang.
I'm an engineer at Jet and hopefully I'll be able to answer your question. The concurrency model within F# is based on continuations. The type Async<'a> = (('a -> unit) -> unit) - its a function which accepts a callback to be notified when the async operation completes. The existing .NET ThreadPool is used to schedule these continuations across OS threads. It uses a trampoline to tame the stack. The ThreadPool itself is a fairly sophisticated piece of work, with scaling heuristics, work-stealing queues, etc. The ThreadPool interacts with the Windows IO completion port multiplexer for IO. We extend this primitive in a variety of ways, notably into an AsyncSeq<'a> which in Haskell terms is ListT Async - a linked list interleaved with Async. We use this for stream processing, sockets, fault tolerance, etc. Async is very similar to Haskell's IO monad, although the representations are a bit different.
However, both Async and IO are insufficient to represent disjunctions. To that end, another concurrency library that we use is Hopac. Hopac is an F# implementation of CML, with some differences. Hopac provides a notion of an alternative (called event in CML; think Haskel's Alternative typeclass if you relax the laws a bit) and synchronous channels (note that async channels are special cases of sync channels). Hopac's has experimental support for lawful MonadPlus as well (see transactional events in Haskell = IO + STM + CML).
Some things that we're heading towards next are generalizing STM to be a bit to be more like RCU (see relativistic Haskell). Additionally, we are experimenting with extending this to session types, but nothing in production yet.
F# also provides a MailboxProcessor, which is similar to an Erlang actor, however without explicit distribution support, so perhaps more of an "agent". We typically use this as a low-level concurrency primitive, rather than a full-blown programming model. Most of our services are compositions of various request/reply interactions, and the Async model above is a great fit for this. In fact, we've primitives centered around the notion of an arrow 'a -> Async<'b> (specialized to Async). These primitives provide support for fault tolerance, logging, tracing, etc.
All of this works well with the GC. Async does cause allocations of course, but this is a price we're more than willing to pay. We've shared some GC dumps with the designer of the .NET GC and she believes they are sensible for a functional language. Hopac took optimization to a greater extreme, reducing allocations where possible.
Another F# library of interest is MBrace. This takes the notion of Async and fits it with a scheduler that schedules across a cluster rather than an individual instance.
Async is similar to a Future (such as in Java), with the difference that Future produces a result once and caches it, whereas Async re-evaluates each time it is executed. It can be made to cache of course. A Future is more like a TPL Task, though IMO, Async provides a more predictable programming model.
Go has go-routines and channels. A go-routine is similar to Async. In essence, they are a notion of light-weigh thread. Note however that Hopac support for channels is far richer than that of Go.
> async channels are special cases of sync channels
You got it the other way around - synchronous is a special case of asynchronous, because any synchronous result or stream can be processed asynchronously, but for having guaranteed synchronous results you're adding restrictions. And going the other way, from async to sync is not possible without blocking threads, which is an error prone, platform specific hack. Take the possibility of blocking threads away and you'll notice the true nature of these models.
In CML/Hopac, async channels (buffers) are implemented in terms of sync channels - there is not async channel primitive built in. Synchronization is the essence of this model. When an operation is waiting on a matching communication through a channel, it is suspended, but no OS thread is blocked.
But yes, going from async to sync requires blocking, which is why CML/Hopac takes the approach of making sync the core primitive.
This is a good point but aren't you doing the same thing? sorry that was a cheeky response.
One thing I will say on your point is that this sort of "side" discourse sometimes brings about real good discussion. Maybe the actual article itself dosent spawn anything new or maybe it is too complex for most folks. However, sometimes the comments lead to new articles to explore a "side" idea that came out from the online comments.
So many articles I've read lately start with "There was a great discussion on hacker news on this topic I felt obligated to write about it, you can find it here".
Look startups aren't perfect. You will most certainly work harder than at a large established firm. Your life balance will be no where near as good as at a large firm.
But why are you electing to work at a startup?
1) You are getting some options and have the possibility of a big upside which does not exist at a large firm.
2) You want more responsibility and you will certainly get it in the form of wearing many hats, but you will work harder.
3) If things are working out you will be promoted faster and move your career forward much quicker than most established firms.
4) There are many other points I could make here but arguably the most IMPORTANT reason to work at a startup is to LEARN, and learn you will.
There are tradeoffs with working at a startup, startups aren't perfect. To say that they are much worse than large corps who are scared of key man risk and keep you at an arms length so they can fire and lay off thousands at a time is disingenuous.
Corporations are generally only after profits big or small. You as an employee have a responsibility to make the right decisions for you.
That's very fair. I can't speak about Amazon, because I've never worked there. You're right that some companies are hanging not on to that startup canon, at least from the outside it seems that way, even after they are large and it's not just Amazon but Amazon is probably the biggest.
My comment generally holds true though that startups have tradeoffs to large corporations. If large corps are trying to take advantage of that then we, as top talent, need to stay away from them so they fail and this trend can go away.
Agreed, a startup has many tradeoffs. For instance, we want to attract key players/talent that we can barely afford. I've been a large contributor to my startup culture over the past 3 years, and while we created an envious work environment, we can only keep the right people that fit the need of the moment. Work hard + play hard really applies to our company and yes it creates some hard times. We live with a constant executive + management pressure to run a lean company and generate profits in our niche market. To me, this is only a mark of respect to the investors that were crazy enough in the first place to believe in our project.
I think that it's hard to understand that chaos unless you've been part of an early startup yourself. I can also understand that 25 years in journalism is a hard fit, it totally goes against the entrepreneurship skills you might need to survive and appreciate the ride in a startup.
It's really not. I worked for a .NET shop that migrated some new projects to Scala. We evaluated F#, but decided the power of Scala was worth the platform switch. F# lacks higher kinds and implicit parameters, meaning you can't really use the type class pattern. This hurt our productivity.
It can be done in F# [1], but it's not pretty, and absolutely nails the compiler. The fact that the compiler can be cheated into doing it makes it even more frustrating that there isn't some official support.
It's just annoying to have to re-implement sequence/traverse and every combination of monad stacks you want to for each type. It's doable, but bug prone and frustrating.
I now work for one of the largest scala teams in the US. We make heavy use of the Scalaz project and regularly use monad transormers or free monads and both require higher kinded types.
Specifically the article missed one of largest F# deployment, in production, in the world at this point. We use F# at Jet.com and it powers every part of our core business from our dynamic pricing algorithm to search and analytics.
Over 4 million customers already on jet and over 2200 cores on azure all running F# code.