API reveals hourly manufacturing activity for every system

Rhivre 2017-05-22

There has been a lot of noise about the regional information available through the Monthly Economic Report and the intel that it gives away to people. There exists, however, a far larger source of intel—the ability to see job hours in Eve on an hourly basis—which has been used by Goonswarm Federation’s Financial Cabal for years to track information across New Eden.

I spent a bit of time (ok, a lot of time) chatting with Probag Bear this week. Probag is the player behind last year’s ‘wall of PLEX’ incident, and one of the people who helped efforts to calculate correct system index multipliers. He provided me with a mammoth spreadsheet of close to 3 million cells with data from across New Eden. He also provided the script for calculating job hours from that data, which has been run by people who know a lot more about coding than me.

This information is available through CREST and ESI, with a bit of calculation added on by the user. I will show some examples of what I mean, and what can be done with this data, although the value of knowing when job hours in a system increase is fairly obvious.

Data Examples

Let’s have a look at some examples first, so you know what we are talking about:

This shows the job hours in The Forge, specifically Jita, for a period between 0540 UTC on May 16 and 1400 UTC on May 16. It shows that between 0640 UTC and 0740 UTC, one thousand job hours were added in the system. This means that during that period (between the CREST pulls) industry jobs totaling 1000 hours were installed. At 0940 UTC you can see that around 1500 hours have been added, this is when early EUTZ people pop in, and AUTZ players are also hitting their peak hours.

So, it shows Jita is busy, so what?

Let’s take a look at another system, this time I am not going to name the region or the system, for reasons that will become obvious.

Job Hours

This is an unnamed system in nullsec, and what is being shown here is that between 1539 UTC and 1657 UTC May 12 someone installed 1279.8 hours of industry jobs. This is around 53 days of industry jobs. It is not a super building, as it is not in Sov null, but, plenty of big things that are not supers can be built.

You can also see that between 1800 UTC and 1902 UTC on the same day another 641 hours of jobs were installed.

This is the same system this morning:

So, between May 13 0700 UTC and May 21 at 0348 UTC, approximately 4622 hours of jobs were installed in this system, which equates to 192 days of building jobs.

Let’s look at a sov null system this time:

 30 Apr 2017 08:30:01 GMT 2557.93
 30 Apr 2017 09:36:12 GMT 3189.3

So, on April 30 between 0830 UTC and 0936 UTC, 632 job hours were installed in this system, which is around 26 days of building.

So what?

Why does it matter if you can see the job hours installed in a system, with a 1-hour resolution? Well, if you know an enemy’s building system, you can monitor it. That lets you see when they put jobs in. If you know how long certain jobs take, then you have a reasonable idea of just what they’re building, as well. So you know what they’re building, where they’re building it, and by knowing when jobs come in, you know when they come out. So you know when to attack to stop those jobs.

Explanation

How does this work? It is actually scarily simple.

CCP provides the cost indices for every system. The cost indices do not however directly give you the job hours in a system, but we know the formula for the calculation from CCP:

So, the calculation is System activity Index = Square root of System job hours/global job hours. It is not quite the System Cost Index, as there is another factor involved, the System Index Multiplier, explained here. This value should remain constant for each system, but can easily be pulled from the API.

The calculation can be modified to be Global Job Hours = (System Job hours/((Cost Index/System Modifier)^2)) or, for a specific system: System Hours = Global Hours*(Manufacturing Index/System Modifier)^2

This leaves the problem of how to find Global Job Hours. That is relatively easy to find. You grab a system where there is no building taking place, drop a POS, install a known number of Job Hours (say, 1000), and wait for the data to show on CREST. What will show up on CREST will be the system cost index, and if you know the System Index Multiplier and the Job Hours you have just installed, you can stick them into the equation.

Once you have global job hours, you can calculate for every system in Eve, every hour.

Limitations of the data

These calculations only show you when something is installed, it does not show when something is delivered, as the calculations for cost index are as follows:

That means that once jobs are registered, they have an effect on the manufacturing index for 28d. There are also some delays in job hours filtering through to the API, usually for systems that have not been used recently. Job hours for these systems filter through 50% at a time over several days, but ultimately, this has little effect on the usefulness of the data.

What now?

The availability of this data is not a bug. It has been available since Crius hit. Until Engineering Complexes hit at the end of last year, though, it was of limited relevance. Now, you have the possibility to explode your enemy’s structures. There are legitimate questions as to whether this level of information should be available to players. Right now, however, it is. There are several people who have the script. Others are prepared to pay well for information about certain regions of null—which should surprise no-one.

There is currently a public API being worked on for this, as well. What that means is that anyone who wants the information will be able to get hold of it. Anyone who wants to will be able to know precisely who is building what, where, and when. All without even needing to undock. The public API is expected to be available within the next week.

Let your voice be heard! Submit your own article to Imperium News here!

Would you like to join the Imperium News staff? Find out how!

Comments

  • Alot

    Its a fairly potent tool for any large organisation that understands whats going on – and an odd piece of information to handout freely. Seems like the sort of thing which should be gamified instead of censored.
    Make the API provide far more detailed and easily understandable data and then make that data available through some sort of in game espionage activity – instead of just downloading it through a public API.

    May 22, 2017 at 8:37 am
    • Caleb Ayrania Alot

      Long have I dreamt of scanning in EVE being a more solid part of intel gathering. Both launched probes and module based scanners. Should really have a history feature, with export option. #scanner-unification-please

      May 22, 2017 at 10:21 am
  • Caleb Ayrania

    The whole industry index thing needs to die in a fire, and be replaced with cost indeces and SOV TAX. That way we would get proper competition pvp and pve, and it would not be “possible” to game without major costs to a SOV holder. The CCP ISK sink would be a cost transaction fee of something in the realm of 1%, the rest should be fully or partially player (supply and demand) driven.

    NB. Worth pointing out that a cost index that players have influence on is more advertising than intel oriented. IF Delve and the Forge show “high cost” that tells no one what is actually being build or who directly benefits.

    May 22, 2017 at 8:47 am
    • Clint Eastwood Caleb Ayrania

      You want industry cheap so players can afford ships.

      May 22, 2017 at 4:21 pm
      • Caleb Ayrania Clint Eastwood

        Nice one line to simplify a much bigger problem. You dont want cheap ships or modules, that only incentive more riskless destruction. Whats next ask CCP to make all prices the same globally, and bounties are so high that everyone can max out the wallets bit limit/goldcap?

        May 22, 2017 at 6:12 pm
        • Clint Eastwood Caleb Ayrania

          And you think this is a problem. Hahahaha. Get wrecked by the Fozz.

          May 24, 2017 at 7:23 pm
  • Moomin Amatin

    This relates back to previous articles on INN regarding the cost of gaining intelligence. The mechanism here is a “no/low effort” solution once the initial investment is made. Of course considerable skill is required to harvest such data and then even more to automate any analysis.

    Anyway a great article and thank goodness that INN is actually finding the real stories in Eve, rather than syndicating tat from forums with the occasional low effort shit post opinion piece.

    Good show Rhivre, looking forward to your next expose.

    May 22, 2017 at 10:17 am
  • Dirk MacGirk

    Was this also raised as an issue by the CSM? Seems far more informative than public concerns about regional information in the MER. In fact, it pretty much negates those concerns so long as this granularity of detail can be deduced.

    May 22, 2017 at 10:33 am
    • Rammel Kas Dirk MacGirk

      I think this is part of an iceberg of information that Aryth gave them at the last summit. You can kind of understand why most CSM’s didn’t quite follow.

      May 22, 2017 at 11:32 am
  • 9T9 RedBalloons

    “There are legitimate questions as to whether this level of information should be available to players”.
    Why should it not be available to players ? The only groups that have an interest to hide this information are the big capship builders.
    StatPr0n is good for the game. it shows to the gaming community as a whole that EvE Online industry “is a thing”. It attracts people to the game that might not be interested in PvP, but in the industrial aspect.
    It shows what fantastic things can be built with the API.
    And it’s just interesting, period.

    May 22, 2017 at 11:42 am
    • There are uses outside supercaps for this data (does somebody build regularly? If so, then they likely ship materials in regularly too. Is there a regular spike at a Sotiyo? Could be fun to blow it up when these materials are in build. It could also indicate a good time to revoke access/offline modules. If it should is available to players, then it should be available as a CREST endpoint as a minimum.

      Statpr0n is indeed amazing, but one of the things people dislike is information being only available out of game. Cost indices are readily available for every system in-game, job hours are not.

      However, I am looking forward to using the API to its maximum when it is released this week, saves me having to go put up a POS somewhere.

      May 22, 2017 at 1:31 pm
    • Erick Asmock 9T9 RedBalloons

      This is terrible information to be available via the API. It goes against everything CCP says they try to instill in the game. Cost, Risk and Effort to do things. There is no in game cost, effort or risk to getting the intel.

      Many things in the API are the roqual of information OPness. ADMs are another issue via the API.

      May 23, 2017 at 1:19 am
      • Dirk MacGirk Erick Asmock

        Not that I think that this level of detail is appropriate, but under your theory of “no in game cost, effort or risk to getting the intel,” that could be extended to even simpler API information related to recruiting like contact lists, emails, ISK transactions and the like, which are used to screen new recruits. There is a plethora of information available from the API that wold otherwise be extremely difficult to come by otherwise.

        May 24, 2017 at 10:17 am
        • Erick Asmock Dirk MacGirk

          I’ll disagree on that point. I equate it to one turning his documentation and records over for security clearance and validating it. There are ways players can use to circumvent that process too. Remember the API and information you are referring to is given ONLY by the player him/herself by choice.

          That is a distinctive difference between that and freely giving other intel information to any player who wants to make mining it easy out of game to anyone. ADM is the most common example but there are others such as the one we are discussing here.

          May 25, 2017 at 1:36 am
  • frankster

    > Anyone who wants to will be able to know precisely who is building what, where, and when.

    This isn’t exactly true – it wouldn’t be possible to distinguish between someone in the same hour installing e.g. 5 fortizars and bit of other stuff, and 2 Thanatos and a bit of other stuff.

    There will be some signatures however that will show up in the logs – e.g. the build time of a single titan researched to any of the 10 levels of TE will be very interesting.

    So for anyone who wishes to conceal their activity as much as possible, my advice would be to set off as many different jobs as possible at the same time in order to obfuscate as much as possible.

    May 22, 2017 at 4:09 pm
    • Rhivre frankster

      I would agree that obfuscation is the best way to hide your activity.

      May 22, 2017 at 5:08 pm
  • frankster

    I would point out that the API is a bit of a red herring here, as its possible to gather this information from within the client.

    May 22, 2017 at 4:26 pm
    • Rhivre frankster

      How do you gather job hours within the client?

      May 22, 2017 at 4:49 pm
      • frankster Rhivre

        Similarly to how you can with the API – get the manufacturing index for your empty test system, install some jobs there and see how it changes the index, then put these values through the formula and you can calculate the global job hours.

        Simultaneously monitor systems of interest via industry window (or the map I believe ) and when you see an index spike, then put this through a similar formula along with your global job hours number, and estimate how many job hours just got installed.

        [ Also, if/when CCP put the titan/super render on sotiyos, you will be able to get this information for any EC you’re near, similarly to how you used to check for POSes with SCSAAs. ]

        May 22, 2017 at 4:58 pm
        • Rhivre frankster

          so you still need to go out of client to calculate, and have a test system.
          you also cant get on the hour every hour indices for all systems in a time efficient way through the client.

          With the API thats going up this week, you dont need to have a reference system set up, or log in to get the info.

          May 22, 2017 at 5:07 pm
          • frankster Rhivre

            Yeah it would no doubt be more convenient to use via the system indexes API, or this 3rd party job hours API if/when it arrives. But its definitely not intel that’s exclusive to API users.

            May 22, 2017 at 5:14 pm
          • Axhind frankster

            It is also the matter of effort. Using in client and going around to check indexes is a lot more effort than API based one. Using effort in game to get intel is perfectly fine. Getting free intel for zero effort is less fine.

            Even if it probably benefits us since delve is so crowded that I doubt that anyone will notice a titan going into build.

            May 24, 2017 at 3:04 am
  • theseconddavid

    If you want to get this out of the game, this is the right way to do it. When people were complaining about the free Intel in watchlists, CcP ignored them for years. Some dude made a way to sideload thousands of contacts at once through crest and CCP fixed it in a few weeks.

    May 22, 2017 at 5:34 pm
    • Break what isn’t broken so someone will fix it. That was how the old adage went, wasn’t it?

      May 23, 2017 at 10:02 am
  • Xrider

    Imperium claims to know who is building supers and titans. Uh, everyone? Deep stuff here. THE SECRET IS OUT!!!!

    May 22, 2017 at 5:39 pm
    • Vertigoe Xrider

      we not only know who is building it but where, when the job was put in and when it will come out.

      May 22, 2017 at 6:33 pm
  • Stephanie Daugherty

    Great example of the power of analytics to infer very sensitive information from data sources that are seemingly innocuous at first glance.

    May 22, 2017 at 9:16 pm
  • Erick Asmock

    CCP needs to review the API. They need a natural progression of what can be derived by the SOV holder vs the rest of the world.

    May 23, 2017 at 1:21 am