UNDERSTANDING THE EVE ONLINE SERVER TICK

2014-07-22

It’s a scenario we’ve all experienced: Your alliance is camping a gate, and a hostile’s jumped through into your camp. Everyone drops drones and assigns them to the fastest locker in the gang, waiting for “point!” to be called on voice comms. The hostile de-cloaks, and it’s a fast ship! Everyone starts locking, and you manage to lock them. But, your point never activates, the game claiming that the hostile is already in warp.

Congratulations: You’re yet another victim of the 1Hz Destiny Server Tick, the processing loop that forms Tranquility’s beating heart.

What IS the Destiny Server Tick?

There’s a few different parts of the Eve Server, but when it comes to shooting things in space, the most important part of them is Destiny. This is the part of the Eve server/engine that simulates grids — it handles advancing the physics simulation of Eve (ships and drones and missiles flying around) and communicating things to the client (physics updates, module activations, target locking, health/status, and other visual elements).

Each solar system, on the server, has a Destiny processing loop associated with it. This loop is always running, and the code inside this loop looks something like this:

  • Pre-Tick Work:
    • Move people between grids, as they warp in/out of grids or cross grid lines.
    • Manage some client actions, mostly those related to piloting and physics: “I want to warp to gate X”, “I want to orbit object Y at 2km”, “I want drone X to return to my bay”
    • Manage some server actions: “Drone X just exploded”, “Object Y just spawned a missile”, etc.
    • Send packets to clients containing everything they need to know to render the game for the next second, including results of actions not on the tick (firing guns, activating tackle, and so on).
  • Physics (aka “Evolve”):
    • For each grid, simulate ships/drones/missiles moving around.  Handle bumps/bounces.
  • Post-Tick Work:
    • Clean up, and go to sleep until the next tick.

The goal of the server is for a server tick to run exactly once per second, or 1 Hertz. If it takes less than a second to do a tick’s work, then everything’s great, and we go to sleep for the rest of the second. (Or, more often, we do a tick for another system on the same node.) But, when it takes longer than that, we have a hard choice to make: either put off some work until the next tick, or come up with more time to process the tick. That’s where Time Dilation comes in; it stretches player activity out so that we have several seconds to do one tick’s worth of work.

Does This Mean Eve Is Actually Turn-Based?

Nope!  Quite the opposite, in fact.

The server tick has some unique limitations to it: It can’t do anything that would require directly talking to most of the database servers, since those can have long lags. (A typical server node handles hundreds of systems, so even though there’s a tick every second for each grid, it might only have 1/100th of a second or less to do all the work for the tick for a single grid. That’s not enough time to talk to a database!)

So most heavy lifting in the code is done by queuing some work to be done on other threads; when that task is finished, the results are reported at the next tick. There’s a long-running urban legend that Eve is single-threaded, and this isn’t true at all; however, the reporting of results is single-threaded.

In fact, Destiny actually does the bulk of its work outside the tick. When most client orders – including “start locking a target” or “turn on a module” – are received by Tranquility, it typically processes them immediately. For example:

  • If you’re trying to jump through gate, it will mark you as “jumping through the gate” the instant that it receives the message, and will send back an acknowledgement right away that you’re jumping. However, everyone else on the grid won’t know that you’ve started jumping until the next server tick.
  • If you activate a turret module, it’ll immediately calculate the damage, and immediately apply it to the target. It’ll send back a packet right away, telling you, “I activated this module for you.”  However, you won’t receive information about how much damage was dealt until the next tick — and neither will the target. Yes, this means you can be dead for up to a second (or up to 10 seconds, in the case of severe TiDi) and not know it yet! This is also how Eve handles weapons that can fire more than once per tick; it simulates them properly, and then both shots appear on the next tick.
  • If you activate a warp scrambler, it’ll immediately tackle the target.  But the “X has warp scrambled Y” global notification won’t appear until the next tick.

Also, some tasks are completely divorced from the tick and can happen at any time, subject only to the latency between you and TQ; for example, starting a scan with probes.

The most important place where this affects you is a pair of closely related actions: aligning/warping and locking a target. In both cases, they get “rounded up” to the next nearest server tick.

Rounding Up: Warping and Locking

When you click “Warp to X” in the UI, your client immediately sends the Eve Server a message: “Align me to celestial X, and warp me as soon as possible.” As soon as it receives it, the server will queue up a work item for the next server tick. At the next server tick, it switches your state from Impulse Mode (normal grid flight) to Pre-Warp Mode. Then, immediately after that, and at each server tick afterward, it’s going to do this check: “Are you above 75% of max velocity, and in the proper direction? If so, immediately switch you to Warp Mode.  Otherwise, continue to move on grid as needed to align.”

Because the check for 75% velocity is done as part of the tick, your align time is effectively ’rounded up’ to the nearest tick — if EFT lists your ship’s align time as 4.5 seconds, the actual time needed to enter warp will vary between 5 and 6 seconds. If this sounds bad, don’t worry; it’s even worse for a would-be tackler.

Imagine that you’re sitting on a gate, with a warp disruptor “primed” – that is, you’ve got it pre-activated and flashing before you have a lock. A hauler jumps through the gate, and you want to lock it and turn on your point. In order, here’s what happens:

  • The hauler’s pilot (let’s say it’s me) clicks “Warp to Jita gate.” Their PC sends a “warp to Jita gate” packet to the server; it arrives, and the work for that warp gets queued for the next server tick.
  • A server tick runs; the hauler decloaks and starts aligning. The server fires off a network packet to your client that tells you, “A Badger piloted by Namamai just appeared on grid at location L, and it’s aligning towards Jita.” It’ll do the same for the target. (Call this moment T+0.) This packet has to cross the Internet to reach your PC — 20-30 milliseconds for European players, 75-80 milliseconds for US players, and 150-200 milliseconds for Australian players.
  • A short time later, the “target decloaked” packet successfully crosses the Internet, and your PC receives the packet and processes it; the Badger appears on overview.
  • Your eyes and brain see the ship appear on the overview, and you click to lock. Your PC sends a network packet to the server: “Please start locking Namamai’s Badger.” Another trip across the Internet!
  • The server receives your “lock target” packet. It immediately processes it and queues a “finish locking” work item that will trigger at the exact duration of your lock time – even if that’s outside of the server tick. It also immediately sends a packet back, acknowledging that you’re starting to lock.
  • Some time passes. No matter what’s happening with server ticks at the time, the “finish locking” timer fires, and the work item executes. It says: “Okay, mark it down that this player is locking that Badger.  At the next tick, tell client X that they’ve successfully locked, and tell client Y that X has yellowboxed them.”  The server stows this information away.
  • At the next server tick — which could happen instantly after, or could happen up to a second later — the server sends that packet to your client, saying, “You locked Namamai’s Badger; it has 70% shields, 100% armor, and 100% hull.  By the way, it’s still aligning towards Jita.” That packet has to cross the Internet!  (At the same time, it’s sending a packet to me to notify me of a yellowbox, and it’s checking to see if I’ve reached the 75% velocity mark to enter warp.)
  • Your PC receives the “lock finished” packet. It sends back a packet saying, “OK, activate my warp disruptor module on X.” Yet another Internet crossing!
  • The server receives your “tackle it” packet. It immediately activates the packet, and queues the “You warp scrambled X” message to be delivered at the next tick. I’m tackled.

That’s a huge list!  Most importantly, it requires four trips across the Internet between your PC and the server.  At a minimum, it rounds up to the next server tick — and it can be worse if you’re on a high-latency connection. From a US player’s perspective, it took them four seconds to lock; adding an additional 100 ms of latency (i.e. moving to Australia) would add a full second to your final point activation time, making it a five-second lock! Specifically, on a high latency connection, your lock may get quantized to the following server tick, adding an extra second before you can activate your point.

Returning to the story at the top of this article, this is what happens when you achieve a lock, but your point doesn’t activate: it means that the ship reached 75% velocity on the same server tick that your lock completed, and it moved into the warping state before you could send the packet to turn on your point.

Practical Application: Instalocking Gatecamps

These rounding effects are particularly painful when it comes to quick-locking ships and quickly-aligning ships. The 1Hz tick of the server produces some nasty thresholds, and given the above information, it’s pretty easy to write a tool that plays with delays in server tick and effects of latency to TQ, and simulates ships trying to lock other ships.

Imagine that we have a Keres with two sensor boosters on it, both scripted for scan resolution. This gives the Keres an advertised lock time for of 1.2 seconds for most interceptors. For comparison, the official align time for most interceptors is between 1.9 and 3.0 seconds, depending on fit and player SP. In theory, the Keres should be able to catch all interceptors with plenty of time to spare. Does it?

In practice, due to latency and server tick rounding, it doesn’t! The threshold is two seconds; any interceptor with an align time of less than two seconds will get away from the Keres. (At best, the Keres will appear to lock them, but the point won’t activate before the interceptor can start warp.)

So, let’s look at nano-fit interceptors. What lock time do we need for the Keres to catch a ship that aligns in 1.9 seconds?  This answer depends a lot on your latency to TQ:

  • With a 100 ms latency to TQ, the Keres needs to be officially able to lock them in less than 0.725 seconds — achievable with three sensor boosters. And, again, this is highly thresholded; anything below 0.725 seconds will catch everything, and anything equal to or above 0.725 seconds will miss everything (due to the point not activating in time).
  • With a 150 ms latency to TQ, the Keres needs to be able to have an official lock time of under 0.625 seconds to lock and point a 1.9 sec-align interceptor.
  • And so on. In general, each 50 ms increase in latency to TQ requires a 100 ms (0.1 sec) reduction in lock time to compensate for it.

To put it bluntly, if you want to catch instawarping interceptors, the most important part is living in London. That, or have a lot of remote sensor boosters — which, unfortunately, are subject to stacking penalties.

Note that this problem is unique to gatecamps; camping station undocks is a slightly trickier problem, as your client is already aware of the ship’s existence and the interplay between initiating lock and undock invulnerability isn’t 100% clear.

Can CCP Fix This?

Ironically, CCP can, and not by nerfing interceptor mobility. Increasing the server tick rate to run twice per second, instead of once per second, completely eliminates most of these distortions — and, far more importantly, it makes playing Eve a much better experience for people on high-ping connections such as Australians / New Zealanders.

The server tick time in TQ is actually a compile time switch; flipping it is as easy as recompiling the server and deploying it. However, don’t expect it to happen any time soon. Doubling the server tick rate means a significant increase in the CPU load for each node. In a world where Jita regularly has TiDi kicking in just from normal player activity, bumping up the server tick rate is probably a non-starter. In order to increase the tick rate, CCP will have to achieve a significant reduction in the amount of work they need to do per tick. (The Brain-in-a-Box/Dogma project is a step in that direction.)

Another way to fix it would be to make priming modules a packet that you could send to the server — it’d eliminate two of the round trips, allowing the modules to be automatically activated by the server on the same tick that the lock finishes.

In the short term, there’s no easy solution. But it’s useful to understand what the server’s actually doing, and why these ships slip away from you. And it’s not just unique to interceptors: pretty much every ship follows this model. Always give yourself a little more scan resolution than you think you need! In particular, you need roughly 2800mm scan res to reliably catch a frigate that aligns in 2 seconds or less.

Thanks to CCP Veritas for his presentation on server ticks, which aided in understanding the tick system.

Let your voice be heard! Submit your own article to Imperium News here!

Would you like to join the Imperium News staff? Find out how!

Comments

  • YT1qKoJ3qQ

    That was a good read.

    April 4, 2017 at 10:05 AM
  • some body

    i done got all smartered upwise when read this here book thing

    June 27, 2017 at 12:26 PM