Market History Data Problems

Rhivre 2017-02-21

The markets in Eve are used by everyone, whether you are dumping your loot to buy orders, building ships, or trading the weekly price swings.  For those who interact with the market for more than instant buys and sells, a key component of their interaction is the market history, which shows you the historical prices for items. There is a problem with this data at the moment, with the values showing in game (and on 3rd party sites) not reflecting the actual history.

Background

There has long been a trimming algorithm on the historical data, which is why those PLEX that sell for 0.01 isk, or 10b isk, do not show up in the data for more than a day or two usually.  This is perfectly fine, and stops the graphs being messed up by outlier prices.

Tritanium with Donchian Channel on

The image above is the graph for Tritanium with all the filters turned on. The price looks flat, and is distorted by an extremely high and low price today. The image below is the same graph with the Donchian channel and min/max turned off

Tritanium without Donchian Channel

The issue with the first graph is caused by the high and low prices in today’s Tritanium price.

Tritanium Price History

A trimming algorithm removes these outliers after they move to historical data, which stops the graphs looking so terrible.

So what is the problem?

The historical data is used for many things, for finding if an item goes to buys and sells for trading, for seeing if the items you build will be able to sell, for speculators looking at trends in prices over time to see if an item is on the way up, or crashing.

The problem is, that this data appears to be incorrect at the moment.  If we take Skill Extractors for example;

Skill Extractors Graph

Extractors History Table

This blank day, occurs on many items, on different days, other examples are:

  • Integrity Response Drones (25 August 2016)
  • Broadcast Nodes (1 September 2016)
  • Nanofactory (27 January 2017)
  • Organic Mortar Applications (7 February 2017)
  • Helium Fuel Blocks (6 January 2017)
  • Concentrated Veldspar (8 July 2016)
  • Veldspar (10 January 2017)

This, in itself, is not a major problem, it is more likely that it is due to something like the data not being logged for those days than for volumes being inaccurate, however, there is a second problem with the history data which is a little more difficult to explain, but, I will give it a try.

Pricing

First of all, an explanation of the numbers on the price history tab.  The orders is the number of instant buy or sell transactions that day, the quantity is the number of items in these transactions. So, if I right click buy 1m Tritanium, it would be recorded as “Orders: 1, Quantity 1,000,000” if I then right click sell 500k Tritanium, it would show as “Orders 2, Quantity 1,500,000” in the price history for Tritanium for the day.  The low price is the lowest price paid that day. So with my Tritanium example, if I right click sell them at 5 isk, it would record that as the lowest price (assuming in this example that I am the only person doing anything with Tritanium, or that I have the first transactions after the date ticked over).  The high price is, the highest price paid that day, so, if I bought my Tritanium at 6 isk it would record that.  The price history tab for the examples laid out above would look as follows:

The average value tells me that there have been more items going from the sell orders than the buy orders so far today. As the average is 33 below the high price and 67 above the low price, it tells me that approximately twice the amount have gone to the high price as to the low price, and that therefore, it is easier today, so far, to sell this item than to buy it.

The low price and high price over time are also useful for tracking trends, and the average will tell you over a given period of time if an item goes to both buys and sells, and in approximately which ratio.

It is in this segment of the price history where outlier removal has the impact, as 0.01 isk PLEX and 1b Tritanium prices are removed here.

Problem

The problem with this data in game, and by extension on 3rd party sites which use CREST history data, is that the pricing information shown is not correct.  As with so many other things in this article, it is easiest to show this with images. The first one is for Genolution CA-4 implants, with the highlighted data being for 19 February.

CA-4 Price History

In the image, you can see that the low, high and average price are all 45 million ISK. What this means is, according to the ingame data, in the 24hr period for 19 February, there were 199 CA-4 traded in the forge, all at exactly the same price.  This might be the case if no one 0.01’d the item, and everyone who did one of the 78 instant buys or sells did it from the same order. This is unlikely in The Forge, as Jita is there. The next image shows wallet transactions from 19 February for CA-4

CA-4 Wallet

This image shows that at least 15 items, on 11 orders went at a price which was not 45 million ISK on 19 February, which does not match with the price history information for the same day.

In the first image, there are several other days visible with low, high and average price all being the same. These are extreme examples, which are easy to spot, and there are many items where you can find examples of this. While for some items, this might be accurate, due to low volume, for high volume items it is incredibly unlikely that all transactions on 1 day in Jita went at precisely the same price.

1mn Afterburner II Price History

In the above image, the data says that 416 transactions, for 1387 items total, all went at exactly the same price in 24 hours in Jita. That means that no one 0.01’d and only one order was bought from or sold to.

Even without all 3 values being the same, there are multiple entries in the table shown that have prices within a few ISK cents of the high, which again, is very unlikely, and there are other price ranges that are extremely close, so that no afterburners would have gone to buy orders at all on that day if the data is accurate.

250mm Railgun II Price History

The same type of data problem can be seen on the image above for the 250mm Railgun II, and on most items in the market.

These entries are clearly unlikely to be accurate, but, the problem with the remaining entries is there is no way of knowing whether or not they are accurate. It is easy to say that the extreme examples are inaccurate, but, they may just be the ones that are easy to spot.  Without keeping a record of the “Today” entry on the price history it is difficult for us as players to know whether or not these values are correct.

The reason for these errors are unknown, they have been present in the price history for at least 18 months, but now the data has got to the point where it is fairly difficult to see whether an item goes to buys or sells with any degree of confidence.  This is not problematic for experienced traders, who already know that for example, high volume T2 modules do go to both buys and sells fairly evenly.  For newer traders however, this obfuscation of data creates problems for them when trying to find items to begin trading with.  It also creates problems for people who are trying to look at trends over a period of time using the graph, or 3rd party tools which display price history.

It might be that the trimming algorithm for removing outliers is being overzealous and cutting prices that are not outliers, or that some other calculations are throwing out the prices, but, whatever the reason, as it stands at the moment, the market history ingame, and the CREST history data are not something that you can rely on with any degree of certainty.  Hopefully, this is something that CCP are able to fix soon, as the economy is one of the unique features in Eve, and many players use the data in various ways for their activities throughout New Eden.

Let your voice be heard! Submit your own article to Imperium News here!

Would you like to join the Imperium News staff? Find out how!

Comments

  • Rammel Kas

    Interesting. But how do we separate the effect of well…. players being what they are and deliberately setting odd prices? The “0” values are one thing and stands out clearly. Lower averages could be something like the Tech bubble or someone trying to do a trick like that too?

    February 21, 2017 at 10:47 AM
    • Rhivre Rammel Kas

      Because you cannot set odd prices in this way.

      In the example with the CA-4 implants, the history says that the high, low and average were the same value on that day, whilst wallet transactions say that items were sold above the high price. On that day, the buys were actually at 44.5 or so, and the sells were the prices shown in the wallet transactions

      You can manipulate the average, but, that is not what is happening here, especially in the examples were all 3 prices are exactly the same.

      February 21, 2017 at 1:26 PM
  • theseconddavid

    This is clearly too much information and should be removed from the game. I demand the CSM get on this right away. Clearly the only fair way to obtain this information is to buy one of everything on the market each day and plot your own graph. Right Goons?

    February 21, 2017 at 12:55 PM
    • Moomin Amatin theseconddavid

      I know that this may be a hard concept to believe but “goons are not here to ruin the game”. Data, in all its forms within Eve, is very important in many respects. Especially in regard to fueling the more interesting stories that come from this sandbox.

      Personally I found this article interesting, especially given that traders have even used the MER historically to better anticipate market movements. The point of this article I think is simply to state that something is not quite working as you would expect. When you talk about daily highs and lows there is an expectation to have some sort of confidence in this. That does not seem entirely unreasonable.

      As for what data is made available and how it can be used for either personal gain or to the detriment of your in game enemies, that is up to CCP. Hopefully the next session of the CSM will be able to represent community issues such as these in a concise manner.

      February 21, 2017 at 4:11 PM
      • theseconddavid Moomin Amatin

        You want tools and data that benefits you and you want to hide data that is a detriment to you. That is human nature. But don’t try to cloak yourself in the narrative that “all information should have a cost”, because there are no goon CSMs running around trying to get killboard and market data shut down. You have CSMs trying to change the game to benefit only your group, which runs completely against the whole purpose of the CSM, which was to keep an eye on things to make sure someone at CCP wasn’t doing things to only help one group.

        February 24, 2017 at 2:31 PM
        • Moomin Amatin theseconddavid

          I work very hard to collect the data that is useful to myself and my coalition, as do many others. If what you were saying was in anyway true do you not think that I would be pushing for better tools? Perhaps even to have our own version of watchlists so that we could add hostiles but not the other way around?

          The proposal by Aryth was put to the CSM which is far from being controlled by The Imperium. If you bother to read the notes you will notice that not a single member objected. Make of that what you will.

          February 24, 2017 at 6:07 PM
  • hurf

    Aryth at it again, damn you goons! reeeeeeee

    February 21, 2017 at 3:42 PM
    • Moomin Amatin hurf

      Have you considered writing an article of your own? I hear that this line of thought is quite popular for some websites nowadays. 🙂

      February 21, 2017 at 4:13 PM
  • Caleb Ayrania

    inb4 #legacycode ! Would be nice if CCP actually dedicated some proper dev hours to the one aspect of EVE that sets it appart from any other mmo game out there. I guess the devs that had a clue how these things worked, and why EVE market was designed as it is, are long gone?

    February 21, 2017 at 4:02 PM
  • jasperwillem

    What about the datadumps in the Economic reports by CCP Quant, dont they fill the gabs?

    February 22, 2017 at 5:51 AM
    • Rhivre jasperwillem

      No, the datadumps in the economic reports are just aggregated values, not individual items

      February 22, 2017 at 2:26 PM