The US Department of Justice has issued guidance to prosecutors regarding compliance programs and their connection to the decision to file charges or recommendations for sentencing. Trade surveillance is one of the tools in the quiver but it isn't the only one. DCM published a whitepaper on the prior April 2019 compliance guidance (int eh May 2019 archive of the blog) that laid out the salient points of consideration. There has been a further update in June 2020 that stressed a couple points:
1. Compliance has to be an evolving practice - if reviews aren't undertaken or or no changes result from reviews, it is likely the program is no really effective;
2. Compliance is not considered effective if it is not adequately resourced - the resource needs are dependent on company scope of activities, level of activities, and size. An under resourced compliance program can be just as much a red flag of lack of corporate importance as having none;
3. If compliance activities (surveillance, training, disciplinary actions) has no impact on staff behavior, it probably isn't working effectively. And if compliance isn't tracking the impacts, the company's commitment to compliance is suspect.
​This brings us back to trade surveillance. Let's ask the first question - why are you adopting it? Is it to have coverage for a future issue or is to truly understand and control behavior within a corporate compliance system. This leads to what DCM would recommend as the process for determining the trade surveillance need.
First, look at what the trading activity is - how many trades, what size, what markets, how many orders are entered per trade (right here is a big stumbling block for companies - they don't record order level activity), how many traders, how many products traded, how many offices, what hours of the day? This is a long list of questions but it boils down to one simple fact - do you know the size of the risk you are trying to control?
Second, once you have the question of where do your risks arise (and if you don't have a former trader or compliance officer to help, make sure the consultant you use does have that experience - a fresh MBA will read a list of risks and a checklist, that will give you an answer but will it give you the right one?), you need to determine what your compliance risk tolerance is. I have seen clients say "we never want any trade to be entered that could cause a concern for the regulator" - my answer to that is "do really want to stop all trading"? There are perfectly legitimate trading patterns and activities that can raise caution flags. The ability to answer the regulator's questions quickly, accurately, and effectively is a huge advantage and should be your goal. Therefore, the minimum bar for compliance, in DCM's opinion, is the ability to identify and resolve potentially suspect activity at a time when memory is still fresh reagrding the strategy, tactics, and actions. That is where trade surveillance comes in - managing that identification and resolution process.
So, the final step will be identifying what types of suspect activity will be observed, what machine learning or AI can be applied to automate some resolutions (but recognize - that automated resolution is going to be a focal point of a regulator if it turns out the automated resolution has been hiding issues, not resolving them), and what resources are needed to manage the more complex resolutions. Automated trade surveillance that is not properly calibrated can generate hundreds of alerts that need to be resolved - failing to resolve them indicates a lack of commitment, exacerbating rather than controlling the compliance issues.
Saying "we need a system" without this analysis can end up creating more costs while also creating greater exposure under the DoJ guidance. Trade surveillance is not a silver bullet - it is a serious solution that can be immensely helpful if deployed at appropriate scale and complexity, it can be a millstone around compliance operations if done ineffectively.

Last week, the DCM blog as on considerations that could influence choosing an "in house" or vendor solution to trade surveillance needs. That discussion really focused on transparent markets (such as futures) much more than physical markets. The closing note was that this week's discussion would focus on physical markets.
The most important questions on choosing an "in house versus vendor data system for physical commodity market trade surveillance are:
​1. What physical products do you trade; and
2. Where do you trade these products?
Fairly simple but crucially important questions that may need some further explanation. The starting point is to understand that all  of the first trade surveillance systems were emerging from the securities markets - equities and such. In those markets, every product has its own singular product code. So, from a data analysis point of view, it is very simple to have all similar trades in the same analysis buckets - just put every trade with the same code in the same bucket. Voila, you have the filtering done. Shifting from equities to futures is just a matter of expanding your look up table to include a bunch of new product codes. And that was the basis for the first energy trade surveillance software (that market started to emerge in the late 2000's with CFTC settlements that required companies to install trade surveillance).  And that is where the roll out hit a screeching halt.
I was SME on one of the very first roll outs. The data model was agile and flexible - the problem was it expected all trades to have a product code. The fun really hit doing global oil where the refined products desk traded US diesel and hitting oil in varying sulfur contents, Europe traded gasoil and, at that time, Singapore was trading products based on centistokes (viscosity). These were all different terminology for nearly the same product. The end result was the necessity to create concatenations of data fields to assemble a "product code". The negotiations necessary to get agreement across desks extended implementation times by orders of magnitude. There were a number of comments on last week's posting that alluded to exactly these questions from others who were also involved in some of these early projects - they have lived through these problems and seen the issues.
Now, fast forward to today. If you trade power and gas in Europe, you likely trade on a Multilateral Trading Facility ("MTF") which has specific requirements under EU regulation. An entity created for regulation of the European energy markets, the Agency for the Cooperation of Energy Regulators - "ACER", published a taxonomy for power and gas markets back in 2015 with the onset of REMIT (Regulation for Energy Market Integrity and Transparency). This led to development of a system for physical energy market product references very similar to futures markets product codes. Voila, the data problem is greatly reduced.
In the US and Asia, however, the natural gas, power, and oil markets have not had the same structure imposed. Therefore, most companies have an idiosyncratic data structure for naming and time bucketing of physical transaction data. Add to that the fact that outside of Europe most regulators do not have access to ongoing reporting of discrete transaction level data as well as any access to order level data for physical natural gas and oil markets (in the US, order level data is available from the Independent System Operators for certain products) and the efficacy of vendor solutions for the physical markets can be severely hampered. 
That efficacy issue does not even address the fact that, unlike exchange markets with products codes, a US physical or power market deployment is likely to be a "build from nearly scratch" data problem that is based on your ETRM system data structure, your naming convention for products in your system, and the complexity of transaction structures you trade.
The metals and agricultural markets can have their own complexities. Think about lending precious metals and the start and end date intricacies of those trades. First, how do you create a consistent product code? Second, what regulator has jurisdiction over these trades and how would they have the data to look at them?
In agriculture, think of the variations in moisture, quality, even age of the grain - you may "simplify" those trades into a common grade and quality for risk control but are they really the same trade for trade surveillance? And, just like in metals, what regulator is looking at those specific trades? Wouldn't a more general in house system that looks at the book as a whole and potential physical/financial cross-market influences be a better fit? Is there a vendor system that thinks that way or is a more simplistic in house build more appropriate?
I still get calls today from trade surveillance software vendors in discussion with customers in US physical markets asking why the customer is being very skeptical regarding their claims to handle physical commodity trade surveillance in the US. I go through a very similar discussion as set forth above. The data issue of "building from scratch" the product codes has a real implication in the estimation of installation costs. As noted earlier, the product code issue can add an additional 4 to 6 to 8 month data reconciliation issue in front of any commencement of the implementation - adding significant hard and soft costs to the customer.
For those reasons, DCM believes that trade surveillance development in physical commodity markets - metals, agricultural products, or energies - needs careful consideration of whether a singular vendor solution for physical and financial products is appropriate or whether bifurcation based on underlying data between vendor and in house might be appropriate.

Having run multiple vendor selection "beauty pageants" for customers as well as designing business rules for major trade surveillance software vendors, the answer is not a simple "of course you do". Because there is a simple fact, trade surveillance software automates finding the needle in the haystack. So, the first question is - do you have a haystack that needs to be sorted?
That is not as "well, duh" a statement as it seems. DCM has consulted with clients where they look at our initial data request and say "what does this have to do with anything?"
For example, a primary question is "how many different products do you trade and how many trades do you execute a week or month - both on average and maximum activity"? Let's see why that is important.
Let's assume a buyer is hedging their supply pricing - all of their purchases are index related and they are buying futures to lock in prices. They execute no more than 100 trades per month which many buy side companies may think as a huge number - I have seen a dozen trades per year for some clients. Let's say they are trading Henry Hub natural gas and two power contract locations. That would mean they execute roughly 33 trades per month per contract. Over 20 business days, that is less than 2 executions per day. That level of activity can be handled in Excel spreadsheets. Even developing simple spoofing models could be done in VBA or Python or something simple. It only would require getting your order level data from your exchange activity (you get one free data feed for this). How much time, effort, and cost is this? Not a lot.
The rationale for a trade surveillance solution becomes much more compelling when you start talking about hundreds of trades per day and thousands of orders per day across twenty or thirty or forty products on three or four or five exchanges. Now, to continue the analogy, you have multiple haystacks in multiple fields and trade surveillance cuts down on the number of people you have to have sorting through the haystacks. If you have a backyard garden, you don't need a John Deere tractor and combine to grow your veggies - if you have 100,000 acres under cultivation, you better have a lot more.
Next week we will talk about the utility of trade surveillance software in physical energy markets (hint - the answer will depend on where you trade these products).

This week's example of "what not to do and how not to do it" comes courtesy of the ICE US Futures Market Regulation disciplinary notices. The story is set forth in ICE notices  here (employee) and  here  (company). The facts are pretty simple:
1. Broker employee screws up customer trades (entered spread trades backwards) as orders and gets filled;
2. Employee panics and tries to reverse trades by doing them as block trades without customer permission or knowledge,
3. Employee does not file block trade documentation
Ok, it seems like a broken record that you have to do block trades according to the rules. It also seems obvious a broker's employees have to know the block trade rules. Well, doesn't seem so in this case. Anyone who has been through training on this would know that you can't get away with block trades without the exchange figuring it out - and, by the way, the employee allocated the trades to push the loss to the customer accounts. Pretty sordid all the way around.
The employee got hit with a $20K fine and a two week suspension form the market for violating Block Trade, improper handling of customer accounts, and "conduct detrimental" charges.
The company got hit with a $30K fine for all the same charges and the normal "failure to supervise" charge - and also paid over $11K in restitution to the customers.
Once again, the company pays more than the employee for the employee's mistake. But the company should have caught the failure to report a block trade and by " failing to properly instruct employees on applicable Exchange Rules."
It frequently comes down to training in these instances. DCM has provided multiple training sessions to firms located outside the US on nit just the content but the culture and reach of US oversight on futures. It is all to frequent when the staff being trained as things like "US exchanges can do that?". Just because you have a good knowledge of your local or regional exchange and regulatory rules and culture, do not assume the US markets are the same. They are not.
​DCM doesn't presume to train people on EU rules, get US training from EU based sources has the same danger.