_______ __ _______
| | |.---.-..----.| |--..-----..----. | | |.-----..--.--.--..-----.
| || _ || __|| < | -__|| _| | || -__|| | | ||__ --|
|___|___||___._||____||__|__||_____||__| |__|____||_____||________||_____|
on Gopher (inofficial)
URI Visit Hacker News on the Web
COMMENT PAGE FOR:
URI Designing a Low Latency 10G Ethernet Core (2023)
up2isomorphism wrote 15 min ago:
I can see the guy is bored to death at Jane Street.
throwaway2037 wrote 9 hours 59 min ago:
LinkedIn tells me: [1] FPGA engineer with a focus on ultra-low latency
networking at Jane Street.
Yikes.
URI [1]: https://uk.linkedin.com/in/ttchisholm
dev_hugepages wrote 7 hours 34 min ago:
Is this really unexpected? A person who knows ultra-low-latency
networking with FPGAs writes an article about making low-latency
networking hardware for a company that needs high-speed, low-latency
communications (high-frequency trading)
anthonj wrote 9 hours 38 min ago:
Not sure i get the implication, something wrong with the company?
exogenousdata wrote 4 hours 43 min ago:
Iâm not the parent, but Iâd assume that this was an impressed
âyikes.â While Jane is not the most well known firm for ultra
low latency trading, an FPGA engineer from that firm could be
considered an expert practitioner in the field.
lloydatkinson wrote 7 hours 29 min ago:
Some people just love dropping âyikesâ type comments and then
never explaining themselves.
userbinator wrote 10 hours 32 min ago:
less than 60ns loopback latency
There are some Ethernet switches with 4ns latency, and those do more
than just sending and receiving, so there's clearly still an order of
magnitude of improvement still available. 4ns is basically ~40 cycles
of the bit clock for 10G Ethernet.
throw0101d wrote 1 hour 29 min ago:
> There are some Ethernet switches with 4ns latency [â¦]
E.g.,
* [1] *
URI [1]: https://www.arista.com/en/products/7130-connect
URI [2]: https://www.arista.com/en/products/7130-series
mgrosvenor wrote 4 hours 26 min ago:
No they donât. At least not on the critical path. L1 switchâs at
4ns simply replicate the incoming electrical signal. Thatâs all
they do.
They also do some packet accounting and cute features off the
critical path. None of that at 4ns.
foobar10000 wrote 2 hours 56 min ago:
In fairness, assuming 4/8 ports in, 4 ports out, operating at some
ungodly GHz using a custom GaAs or SiGe chip, and working in
gearboxed scrambled space with very clever input mac prefix mapping
and output scrambled/gearboxed precomputation, one _could_ do
around 8ns optimistically from the start of the first 64b/66b block
(the 0x55... preamble) for 10GB. There's some stuff about preamble
shrink that makes it wonky, but that is doable. And interestingly
enough, for 25G-R, one can comfortably do this in about 4ns. I am
not in fact aware of such a beastie existing for 25G, but I have
seen 1 or 2 for 10G though.
Surprisingly, if ChatGPT is prompted _juust_ right, it will even
give you a good way to do this.
rrr_eady wrote 4 hours 17 min ago:
This guy knows a thing or two about these devices ;)
foobar10000 wrote 3 hours 23 min ago:
Yes he very much does :)
darrin wrote 4 hours 56 min ago:
There are L1 crosspoint "switches" in the 2-5ns range depending on
port density and similar modes in some Ethernet silicon. These are
not Ethernet switches in any normal sense though. They only replicate
the signal 1:1 or 1:n and do not dynamically switch the destination
based on anything in the packet. The fastest L3 Ethernet switches on
the market are ~90ns.
jacquesm wrote 5 hours 8 min ago:
It's a bit more complicated than that. Loopback implies that the data
went off the wire and was sent back. That puts some serious
constraints on the process that deals with that data, how it is
stored and how it is fed back into the system. You can't just compare
that with naked pass-through latency (which is what a switch would
do), because the signal is already on the way out before it has even
fully been received.
Essentially a few ns after the header is received it will already be
passed on to the output port after some signal conditioning, which
happens almost without latency. A typical 'high speed' switch will
have 800 ns round trip, so that 60 ns quoted here is actually quite
impressive.
rrr_eady wrote 6 hours 37 min ago:
For switching, this is just not true. There's in/out solutions that
are at this kind of scale, but they are by definition not
implementing any switching logic.
rasz wrote 8 hours 53 min ago:
Cut Through switching needs what, 14 bytes? Thats ~12ns on 10G? How
would you do it in 40 cycles?
ongy wrote 7 hours 25 min ago:
why do you assume the full ethernet header?
IIRC. cut-through only needs the first 6 bytes. Since it only needs
the destination address for the port lookup.
Potentially the first bit, on broadcast.
rasz wrote 6 hours 18 min ago:
Because you might want VLAN. Plus you cant just start blasting
reply without a preamble, so its still 14 bytes after receiving
just the dest MAC. Then you get potential IFG, FEC, scrambling,
it all adds up, no way any switch can do 4ns without heavy lawyer
talk in the small print.
Perz1val wrote 8 hours 35 min ago:
Not my are of expertise, but afaik the neat part is that you don't
do it in cycles. The "program" lives as the transistor layout, not
as instructions
wtallis wrote 8 hours 32 min ago:
The cycles in question are not those of any processor, but those
of the signal coming in the wire. No matter how it's implemented,
a switch cannot react to bits that are still in the wire and
haven't made it to the switch yet.
mrlongroots wrote 9 hours 18 min ago:
The other funny bit is that one-way PCIe latency is 250ns-ish (don't
quote me on the exact numbers), which imposes a hard 1us constraint
on latency between two hosts.
mgrosvenor wrote 4 hours 25 min ago:
You can go quicker with CXL, but not by much.
foobar10000 wrote 3 hours 57 min ago:
There are always DPUs like NextSilicon, NVidia's Blue*, etc - at
under 100 ns of SoF to fast compute though.
Neywiny wrote 13 hours 19 min ago:
It seems fun to be a high frequency FPGA trader designer. All my FPGA
is much lower power consumption so I don't get to play with stuff like
gigs of external SRAM or the QDR stuff or whatnot
amelius wrote 4 hours 35 min ago:
> It seems fun to be a high frequency FPGA trader designer.
Yes, if you think that doing pointless work is fun. What do you tell
your grandchildren?
KeplerBoy wrote 2 hours 32 min ago:
Meh, at least HFT is making sure markets are efficient.
Now imagine telling your grandkids you worked on pushing short form
content in absolutely every app to maximize engagement and rot
brains?
bombcar wrote 3 hours 2 min ago:
Will the grandchildren be able to hear you over the roar of the
yacht?
jbstack wrote 4 hours 13 min ago:
It's a bit unfair to call it pointless. Empirical studies have
shown that HFT increase liquidity in markets, leading to narrowed
spreads, market depth, faster execution speeds, cross-market
informational efficiency, etc. All of these are generally
beneficial to other non-HFT market participants.
Now you might counter that it also brings some negatives (e.g.
fleeting liquidity, market manipulation risk, etc.) and there's
certainty some validity to that. But none of this supports an
argument that it's pointless.
More generally, activities aren't pointless if you are earning
money from it which pays your rent, puts food on your table, etc. A
large portion of the world works mainly for this goal. It's
perfectly valid to tell your grandchildren "I spent my life making
sure we'd have enough money for you and your parents to live
comfortably", and we shouldn't look down on someone who does that.
gosub100 wrote 10 min ago:
It's pointless in terms of the skill wasted that could be used
for actual cool things. But I agree that more market participants
is better. And whether I'm buying or selling, I always want
someone to trade with, vs not.
Retric wrote 2 hours 55 min ago:
> HFT increases liquidity
On average with a few caveats yes, but it also causes issues by
reducing the vastly more important signaling aspects of the
market.
Trade volume for example can get almost completely divorced from
economic reality.
amelius wrote 4 hours 10 min ago:
> Also, it's not pointless to the person actually engaging in HFT
if they are making money from it. Activities aren't pointless if
they pay your rent, put food on your table, etc.
You have to look at the bigger picture. HFT companies only
increase the barrier to entry to participate in the market. This
means that while you may be able to put food on the table, you
make it more difficult for other people to put food on the table.
OneDeuxTriSeiGo wrote 3 hours 53 min ago:
Markets should not primarily be a casino.
If you want to invest some of your income into a spread of
companies? Go for it. But if all you are doing is trying to
game the market? That's just gambling, go to a real casino and
have some fun instead.
HFT should be driven by these types of market makers exactly
for the purpose of decreasing the amount of unnecessary
speculation and volatility in the markets. HFTs killing the
spreads is one of the only reasons the markets didn't implode
during covid despite the extraordinary fear and volatility that
was impacting them.
You want a parasite? Look to hedge funds who just sit on piles
of other people's cash to scoop obscene amounts off the top
(relative to their actual contribution) for occassionally
shuffling funds between financial instruments.
amelius wrote 3 hours 42 min ago:
> HFTs killing the spreads is one of the only reasons the
markets didn't implode during covid despite the extraordinary
fear and volatility that was impacting them.
And why would this not be possible if everybody traded at the
same, low, frequency? Alternatively, what is the minimum
trading frequency required to save markets from collapsing?
By trying to answer these questions you see how ridiculous
the situation with HFT really is.
Galanwe wrote 1 hour 58 min ago:
Sorry but I don't understand any of your arguments.
You are throwing things like "HFT increase the barrier to
entry", "HFT make things unfair", but... how so?
What would _you_ do if you had the capability to trade at
the speed of light while others cannot?
Trading faster does not magically make money appear in your
account, it does not magically tell you what is going up
and what is going down.
It seems people work under the assumption that HFTs make
money by having the same ideas than others, but trading it
faster. That is _not_ what market making is, that is _not_
what HFTs do, and trading 10ns "faster" is of absolutely no
use to 99.99% of market participants. In fact, the vast
majority of market participants purposefully *slow* their
signals to avoid impacting the market. At any given point
in time, the orderbook contains but a tiny fraction of the
liquidity that most people are willing to trade.
HFTs usually run on a combination of 4 businesses, none of
which provide any downside for the rest of the
participants: Market making, events signals, order flow and
arbitrage.
Market making: essentially they are paid (or more
precisely, rebated) to "animate" the market (provide
liquidity). Everyone is happier with tight spreads, and
everyone is happier with some level of liquidity at the top
of the book, I don't really see how that could be up to
debate. This is a tough business, the market maker has to
provide a contractual amount of inventory for buyers and
sellers at the same time, with the risk that these assets
can inadvertently increase or decrease in value. Market
makers get a premium (the spread) in exchange for providing
liquidity to market takers, taking the risk of holding that
inventory for them.
Order flow: HFTs are usually more knowledgeable than
managers (and even brokers) to perform trade execution.
Remember, the price of an asset does not move "by itself".
What makes the price of an asset _really_ increase is an
imbalance between the size of buys and sells, causing the
spread to be crossed increasingly in one direction, and
eating higher and higher price levels => a market
participant buying shares is self realizing the price
increase. That is called market impact, and that is
something nobody wants. In an ideal world, nobody would
know the actual liquidity that other participants are
willing to exchange on the market, that way no arbitrage
would be possible, no market impact, everyone is happy. The
problem though is that you need _some_ level of visibility
otherwise you hinder so called "price discovery", and
regulators don't like that because it's an open door for
scams, hidden fees, etc. So a lot of investors just
delegate the actual order passing to HFTs: you agree on a
price you're happy with, and the HFT executes the orders
for you. If they manage to get a better price: good for
them, they can pocket the difference. If they didn't, they
will have to eat the cost and do better next time. Again,
it's a _difficult job_, because by very definition managers
are going to try to give you _toxic flows_: if someone
mandates you to buy 100 shares of Microsoft, it's because
they think the price will go up, but you will quote them at
the current stock price. I don't think anyone is negatively
impacted by that business to be honest. I am quite happy to
instantly do a trade at a known price and let an HFT take
the risk of executing trades, finding liquidity, etc.
Arbitrage: This may be the most well known HFT strategy.
Essentially benefiting from price disparity of more or less
fungible assets. So if HSBC is priced the equivalent of
$100 on the LSE and $101 on NYSE, they buy there and sell
here. I don't think there is much debate on whether that's
a good thing or not, everyone should be happy that prices
are more or less consistent around the world. Most people
also wouldn't want to bear the overnight nor settlement/FX
risk of holding these.
Event signals: I guess that's what people overreact to,
even though this is but a tiny and negligible part of the
HFT businesses. There are strategies where the whole
concept is just to be the first to react to some kind of
event, and yes HFTs are the best placed to do such
strategies. The thing is, this is not a very lucrative
business for a simple reason, which I already stated
before: at any point in time, the orderbook contains but a
mere fraction of the available liquidity. And it takes
*ample time* for the market impact of a trade to be
absorbed. There are a lot of papers on the subject, but
without entering in too much details: the bigger your
order, the longer it will take for the book to absorb it
and liquidity to come back. For most hedge funds doing
statistical arbitrage, CTAs, etc, we are talking multiple
*days*. So realistically, if you're in the business of
reacting the fastest to some kind of news or event, there's
really not much more you can do than cross the spread and
eat one or two levels of the book. Even for the most liquid
of US stocks, that's a couple hundred thousands dollar per
trade on average, for a tiny return difference versus
executing a couple of seconds later, so I hope you find a
lot of these if you plan on getting rich.
TLDR, I think 99% of HFT hate is completely unjustified.
They provide a legitimate service to most market
participants, and as an investor I wouldn't want to do
their job, nor would I think that they hinder in any way my
investments, on the contrary.
cap11235 wrote 51 min ago:
I think it would also be useful to distinguish between
high frequency and low latency trading in these sorts of
discussions. The groups of traders that do actual high
frequency are not the same ones that are doing
shenanigans with things like speculative Ethernet frame
invalidation (at least on the same engine).
amelius wrote 52 min ago:
Can you explain why we need nanosecond trading for this?
Why can't we use trading speeds such that everybody can
participate, not just a few "gatekeepers"?
The average internet connection should be fast enough for
trading, by now.
jbstack wrote 4 hours 4 min ago:
How's that? HFT lowers the barrier of entry for ordinary
participants who just want to invest their savings either
directly or through something like a pension fund, because they
get all those benefits I mentioned, and those benefits are
useful when you are only executing a trade once a month or
year.
It only increases the barrier to entry to people who want to do
HFT themselves on a smaller scale, because they now can't
compete on latency etc. But it's a bit of a circular argument
to say that it's a bad thing that engaging in HFT increases the
barrier of entry to HFT.
In any case, and to reiterate, I wasn't refuting a claim that
HFT is a bad thing, I was refuting your original claim that
it's pointless. For something to be pointless, there has to be
no purpose to it whatsoever, which means it has to have no
benefits.
amelius wrote 3 hours 48 min ago:
We all know that middlemen are bad.
We don't need an HFT middleman.
gricardo99 wrote 1 hour 29 min ago:
actually you do need another participant to take the other
side of your trade. Thatâs not a middle man. Market
makers, liquidity providers, are a key component of these
markets, taking risk with their own capital (no systemic
risk, no too big to fail, going to get a government is
things go badly), requiring sophistication and
specialization to survive in an ultra competitive
environment with a high degree of uncertainty and risk
exposure.
le-mark wrote 4 hours 27 min ago:
High frequency trading is far from the only pointless activity
software and hardware developers are employed to do. At least in
that realm they are pushing the state of the art, unlike presenting
ads or yet another crud app.
amelius wrote 4 hours 12 min ago:
Pushing the state of the art? That's only meaningful if the
results are published. But I suspect that everything is kept
secret.
Come on, it is not difficult to find better things to do with
your brains. Like work in medicine, etc.
OneDeuxTriSeiGo wrote 4 hours 1 min ago:
You do realise that Jane Street (one of the big HFT market
makers) publishes basically all their technical work and
contributes to massive amounts of open source software? They
are to my knowledge one of the main contributors to the OCaml
ecosystem as well.
mungoman2 wrote 10 hours 40 min ago:
I agree! Seems like very interesting work.
typpilol wrote 12 hours 44 min ago:
I feel like it's also got to be super stressful
Imagine getting a call you system is down and is costing them
millions per hour
zorked wrote 6 hours 20 min ago:
The way to think is the opposite: your system is making millions
per hour.
ktm5j wrote 5 hours 26 min ago:
What about when something goes wrong because you overlooked some
detail?
jacquesm wrote 5 hours 11 min ago:
Shit happens. I wrote the firmware for a multi-link HDLC card
which earned vast amounts of money for the company that
commissioned it. I was quite aware of my responsibility and so
was the hardware team. We stress tested the crap out of it
before releasing to production and fortunately it never locked
up when it was in actual use but we had some pretty wild and
very rare (so hard to trigger and reproduce) bugs which delayed
deployment considerably. But none of this gave me the kind of
feeling that working on fuel estimation software for aircraft
gave. That's when there are in the most literal sense lives on
the line, and that's a completely different kind of pressure.
You simply can not fuck up. It also really rammed home the
value of code review and having a good specification so that
you can ensure that within the envelope of input parameters
your software does what it is supposed to do.
drmpeg wrote 4 hours 15 min ago:
Multi-link HDLC. That brings me back to 1985 and the Intel
8274, Zilog Z8530 and the Western Digital WD2511 (that
implemented the layer 2 protocol in silicon).
jacquesm wrote 4 hours 3 min ago:
Yes, that was around that time. There were a couple of
problems, the first batch of chips we got was very early in
the development stage of the chip, pre-production issues
and there were some bugs in the chips themselves which
could cause lock-up under some circumstances. We found ways
to work around those and then of course there were all of
the niceties around dealing with a device that generates an
extremely high rate of interrupts. So the code would either
have to attempt to service all channels on any interrupt or
be re-entrant. I don't remember which solution we picked
but in the end the thing was, once we had the bugs worked
out fairly bullet proof.
One funny bit about the development process was that
initially I was going to be nice and implement every layer
as its own stand-alone bit of software communicating via a
defined set of primitives with the layers above and below.
But that was slow as molasses so in the end all of that
elegance got discarded for an absolutely unholy sandwich
that did all of the layers in a single chunk of code. But
with the experience from the 'slow' version that was
actually doable, I would have never been able to write that
as the first implementation. Classic illustration of 'first
make it work, then make it fast'.
drmpeg wrote 3 hours 3 min ago:
Cool. We had a small company in San Diego called Metacomp
(long gone now) design a Z8530 add on module for their
80186 based Multibus board. It had two Z8530 chips on it
and you could have two modules on the base board. So 8
channels total per slot.
jacquesm wrote 2 hours 48 min ago:
Yes, 8 was just about the maximum, simply because that
would approach the limits of how many interrupts you
could process before you'd inevitably start losing
them. Also, space and power delivery limitations, and
connector space on the backside made doing more than
that very impractical. Or you had to take more than one
interrupt per board but that was 'not done'. I've
always loved working on that division line between
hardware and software, as close to the metal as
possible. Funny, I'd all but forgotten about this
project, now I'm wondering if I still have the software
somewhere, just searched for a bit and can't locate it,
so there is a good chance I wrote it on company
hardware. I did find another project that I'd forgotten
about, a CAD program for sails for sailboats that I
wrote in the 80's for TD Sails in the Netherlands. That
was a great project to work on and I met some awesome
people doing it.
hsbauauvhabzb wrote 5 hours 47 min ago:
No the way to think is âitâs down less than our closest
competitors systemâ
ta1243 wrote 9 hours 20 min ago:
I'd want to know why they had such a system as a single point of
failure
hshdhdhehd wrote 5 hours 47 min ago:
Failure modes can be a bug making a bad trade while doing so at
100% uptime.
kakacik wrote 7 hours 10 min ago:
No business is designed ideally. Now do you want the job or we
move to another one who will accept this?
toast0 wrote 12 hours 21 min ago:
On the other hand, no nights or weekends.
noitpmeder wrote 11 hours 30 min ago:
This is becoming less and less true every day. 24/7 markets are
coming whether you want them or not, and if you're not trading
outside of US Equity hours you're leaving significant money on
the table.
Galanwe wrote 9 hours 33 min ago:
> This is becoming less and less true every day. 24/7 markets
are coming whether you want them or not
This has still to exist for real. Every couple of years there
is a resurgence of "we should have 24/7 equity markets",
followed by milestone announcements of Nasdaq.
The reality is more... nuanced.
First, there has to be a wide enough window for corporate
actions and news dissemination to happen. Not only is that a
regulatory requirement, but anyway no sane investor would like
to trade in such a window even if possible.
Second, there are already early sessions, pre opening sessions,
and late sessions on most American markets. The liquidity there
is almost inexistant, so I wouldn't say there is a huge demand
for extended trading hours by non-retail participants.
And last, half the liquidity of US markets is not on lit books,
and half the remaining lit liquidity is close to the auction. I
don't think it would even be a net profit for lit exchanges to
extend their hours in these conditions. As for dark pool, I
think only 2 or 3 of the smaller ones are 24/7, like the one
from IB.
> if you're not trading outside of US Equity hours you're
leaving significant money on the table
HFTs need a lot of liquidity and tight spreads, that's pretty
much only doable at scale in the US and Japan. Europe and EM
are either not liquid enough, or only really able to sustain
low capital ad-hoc strategies.
dfex wrote 11 hours 2 min ago:
Genuinely curious - would 24x7 low-latency trading matter then?
Wouldn't after-hours trading (for your local timezone) be
happening on an exchange in another location?
Or do you mean dark pools/private exchanges that may run 24x7?
hamonrye wrote 12 hours 14 min ago:
PoE is the way to use low voltage in electronics.
DIR <- back to front page