Becoming really rich with C#

Luca Bolognese - Sep 2009

Or maybe not, please do not hold me responsible if you lose money following this system. Having said that, it is my opinion that there are very few concepts that are important in investing. Three big ones are value, diversiﬁcation and momentum. This post is about the latter two and how to use C# to create a simple trading system that uses both.

Diversiﬁcation is ‘not put all your eggs in one basket’ (contrary to ‘put all of them in one basket and watch that basket’). I don’t believe you can ‘watch’ very much in ﬁnancial markets, so I tend to prefer diversiﬁcation.

Momentum is a mysterious tendency of ﬁnancial prices that have risen the most in the recent past, to continue outperforming in the close future. In essence, buying the top stocks/sectors/asset classes tends to outperform buying the bottom ones over horizons from three months to one year.

The idea then is to rank some assets (i.e. ETFs) by how fast they have risen in the past, go long the top ones and short the bottom ones. There are hundreds of variations of this basic strategy, we’ll add the rule that we won’t buy assets that are below their 200 days moving average or sell short assets that are above it.

I’m writing this code with VS 2010 Beta 2 (which hasn’t shipped yet). It should be trivial to modify it to run on B1 (or maybe it does run on it already). I attach the code and data ﬁles to this post.

struct Event {
    internal Event(DateTime date, double price) { Date = date; Price = price; }
    internal readonly DateTime Date;
    internal readonly double Price;
}

We’ll use this simple structure to load the closing price for a particular date. My use of internal is kind of bizarre. Actually the whole code might look strange. It is an interesting (maybe un-elegant) mix of object orientation and functional programming.

class Summary {
    internal Summary(string ticker, string name, string assetClass,
                    string assetSubClass, double? weekly, double? fourWeeks,
                    double? threeMonths, double? sixMonths, double? oneYear,
                    double? stdDev, double price, double? mav200) {
        Ticker = ticker;
        Name = name;
        AssetClass = assetClass;
        AssetSubClass = assetSubClass;
        // Abracadabra ...
        LRS = (fourWeeks + threeMonths + sixMonths + oneYear) / 4;
        Weekly = weekly;
        FourWeeks = fourWeeks;
        ThreeMonths = threeMonths;
        SixMonths = sixMonths;
        OneYear = oneYear;
        StdDev = stdDev;
        Mav200 = mav200;
        Price = price;
    }
    internal readonly string Ticker;
    internal readonly string Name;
    internal readonly string AssetClass;
    internal readonly string AssetSubClass;
    internal readonly double? LRS;
    internal readonly double? Weekly;
    internal readonly double? FourWeeks;
    internal readonly double? ThreeMonths;
    internal readonly double? SixMonths;
    internal readonly double? OneYear;
    internal readonly double? StdDev;
    internal readonly double? Mav200;
    internal double Price;
    internal static void Banner() {
        Console.Write("{0,-6}", "Ticker");
        Console.Write("{0,-50}", "Name");
        Console.Write("{0,-12}", "Asset Class");
        //Console.Write("{0,-30}t", "Asset SubClass";
        Console.Write("{0,4}", "RS");
        Console.Write("{0,4}", "1Wk");
        Console.Write("{0,4}", "4Wk");
        Console.Write("{0,4}", "3Ms");
        Console.Write("{0,4}", "6Ms");
        Console.Write("{0,4}", "1Yr");
        Console.Write("{0,6}", "Vol");
        Console.WriteLine("{0,2}", "Mv");
        //Console.Write("{0,6}", "Pr");
        //Console.WriteLine("{0,6}", "M200");
    }
    internal void Print() {
        Console.Write("{0,-6}", Ticker);
        Console.Write("{0,-50}", new String(Name.Take(48).ToArray()));
        Console.Write("{0,-12}", new String(AssetClass.Take(10).ToArray()));
        //Console.Write("{0,-30}t", new String(AssetSubClass.Take(28).ToArray()));
        Console.Write("{0,4:N0}", LRS * 100);
        Console.Write("{0,4:N0}", Weekly * 100);
        Console.Write("{0,4:N0}", FourWeeks * 100);
        Console.Write("{0,4:N0}", ThreeMonths * 100);
        Console.Write("{0,4:N0}", SixMonths * 100);
        Console.Write("{0,4:N0}", OneYear * 100);
        Console.Write("{0,6:N0}", StdDev * 100);
        if (Price <= Mav200)
            Console.WriteLine("{0,2}", "X");
        else
            Console.WriteLine();
        //Console.Write("{0,6:N2}", Price);
        //Console.WriteLine("{0,6:N2}", Mav200);
    }
}

The class Summary above is how I want to present my results. A few comments on the code. I use Nullable because some of this values can be null (i.e. not enough history), but I still don’t want to worry about it. It ends up working rather neatly.

I also print the results out to Console, which is crazy. I really should be using WPF/Silverlight as the presentation layer. Also the {0,4:N0} notation might be unfamiliar to some of you, but this is how mad Console guys like myself avoid using real UI frameworks. Sometimes we print things in color too.

The real meat is in the following line:

LRS = (fourWeeks + threeMonths + sixMonths + oneYear) / 4;

That is our highway to richness. It’s a very elaborated quant formula, never before shown, that calculate a magick relative strength (aka momentum) factor as the average of the performance of four weeks, three months, six months and one year.

class TimeSeries {
    internal readonly string Ticker;
    readonly DateTime _start;
    readonly Dictionary<DateTime, double> _adjDictionary;
    readonly string _name;
    readonly string _assetClass;
    readonly string _assetSubClass;
    internal TimeSeries(string ticker, string name, string assetClass, string assetSubClass, 
                                                                IEnumerable<Event> events) {
        Ticker = ticker;
        _name = name;
        _assetClass = assetClass;
        _assetSubClass = assetSubClass;
        _start = events.Last().Date;
        _adjDictionary = events.ToDictionary(e => e.Date, e => e.Price);
    }

I then built myself a little TimeSeries class that represents a series of (date, price). I choose a dictionary to store it because of my assumption that I will be accessing it by date a lot. In retrospect, I was kind of right and kind of wrong. It doesn’t really matter much.

bool GetPrice(DateTime when, out double price, out double shift) {
    // To nullify the effect of hours/min/sec/millisec being different from 0
    when = new DateTime(when.Year, when.Month, when.Day);
    var found = false;
    shift = 1;
    double aPrice = ;
    while (when >= _start && !found) {
        if (_adjDictionary.TryGetValue(when, out aPrice)) {
            found = true;
        }
        when = when.AddDays(-1);
        shift -= 1;
    }
    price = aPrice;
    return found;
}

A TimeSeries can give you back the price at a particular date. This looks bizarre and complex, but there is a reason for it. I might ask for a date that doesn’t have a price associated with it (i.e. holidays, week-ends). In such cases I want to return the previous price which could be N days in the past.

I also want to return how many days in the past I had to go, so that other calculations (i.e. Return) can modify their end date by the same amount. Also I might not ﬁnd such a price at all, in which case I don’t want to throw an exception, but instead notify the caller. In retrospect, I should have used double? to signify ‘price not found’.

double? GetReturn(DateTime start, DateTime end) {
    var startPrice = 0.0;
    var endPrice = 0.0;
    var shift = 0.0;
    var foundEnd = GetPrice(end, out endPrice, out shift);
    var foundStart = GetPrice(start.AddDays(shift), out startPrice, out shift);
    if (!foundStart || !foundEnd)
        return null;
    else
        return endPrice / startPrice - 1;
}

We can now go and calculate the return between two dates. Also the TimeSeries object needs to perform a little more calculations.

internal double? LastWeekReturn() {
        return GetReturn(DateTime.Now.AddDays(-7), DateTime.Now);
    }
    internal double? Last4WeeksReturn() {
        return GetReturn(DateTime.Now.AddDays(-28), DateTime.Now);
    }
    internal double? Last3MonthsReturn() {
        return GetReturn(DateTime.Now.AddMonths(-3), DateTime.Now);
    }
    internal double? Last6MonthsReturn() {
        return GetReturn(DateTime.Now.AddMonths(-6), DateTime.Now);
    }
    internal double? LastYearReturn() {
        return GetReturn(DateTime.Now.AddYears(-1), DateTime.Now);
    }
    internal double? StdDev() {
        var now = DateTime.Now;
        now = new DateTime(now.Year, now.Month, now.Day);
        var limit = now.AddYears(-3);
        var rets = new List<double>();
        while (now >= _start.AddDays(12) && now >= limit) {
            var ret = GetReturn(now.AddDays(-7), now);
            rets.Add(ret.Value);
            now = now.AddDays(-7);
        }
        var mean = rets.Average();
        var variance = rets.Select(r => Math.Pow(r - mean, 2)).Sum();
        var weeklyStdDev = Math.Sqrt(variance / rets.Count);
        return weeklyStdDev * Math.Sqrt(40);
    }
    internal double? MAV200() {
        return _adjDictionary
               .ToList()
               .OrderByDescending(k => k.Key)
               .Take(200)
               .Average(k => k.Value);
    }
    internal double TodayPrice() {
        var price = 0.0;
        var shift = 0.0;
        GetPrice(DateTime.Now, out price, out shift);
        return price;
    }
    internal Summary GetSummary() {
        return new Summary(Ticker, _name, _assetClass, _assetSubClass, 
                           LastWeekReturn(), Last4WeeksReturn(), Last3MonthsReturn(),
                           Last6MonthsReturn(), LastYearReturn(), StdDev(), TodayPrice(),
                           MAV200());
    }
}

Nothing particularly interesting in this code. Just a bunch of calculations. The MAV200 is the 200 days moving average of closing prices. It shows a more functional way of doing things. The StdDev function is instead very imperative.

We now can work on downloading the prices. This is how you construct the right URL:

static string CreateUrl(string ticker, DateTime start, DateTime end) {
    return @"http://ichart.finance.yahoo.com/table.csv?s=" + ticker + "&a="
            + (start.Month - 1).ToString() + "&b=" + start.Day.ToString() + "&c="
            + start.Year.ToString() + "&d=" + (end.Month - 1).ToString() + "&e="
            + end.Day.ToString() + "&f=" + end.Year.ToString() + "&g=d&ignore=.csv";
}

And let’s set how many concurrent connections we are going to use …

ServicePointManager.DefaultConnectionLimit = 10;

On my machine, setting this number too high causes errors to be returned. I’m not sure on which side of the connection the problem lies.

We can then load all the tickers we want to load from a ﬁle. One of the ﬁles has Leveraged ETFs, which I want to ﬁlter out because they tend to pop up always at the top.

var tickers =
    //File.ReadAllLines("ETFs.csv")
    //File.ReadAllLines("ETFTest.csv")
    File.ReadAllLines("AssetClasses.csv")
    .Skip(1)
    .Select(l => l.Split(new[] { ',' }))
    .Where(v => v[2] != "Leveraged")
    .Select(values => Tuple.Create(values[], values[1], values[2], values[3]))
    .ToArray();
var len = tickers.Length;
var start = DateTime.Now.AddYears(-2);
var end = DateTime.Now;
var cevent = new CountdownEvent(len);
var summaries = new Summary[len];

And then load all of them, making sure to make an asynchronous call so not to keep the thread busy.

for(var i = 0; i < len; i++)  {
    var t = tickers[i];
    var url = CreateUrl(t.Item1, start, end);
    using (var webClient = new WebClient()) {
        webClient.DownloadStringCompleted +=
                          new DownloadStringCompletedEventHandler(downloadStringCompleted);
        webClient.DownloadStringAsync(new Uri(url), Tuple.Create(t, cevent, summaries, i));
    }
}
cevent.Wait();

Notice the use of a Countdown event to wait for all the thread to complete before printing out the results. Also notice the new Tuple class used to package things to send around.

We can then print out the top and bottom 15%:

var top15perc =
        summaries
        .Where(s => s.LRS.HasValue)
        .OrderByDescending(s => s.LRS)
        .Take((int)(len * 0.15));
var bottom15perc =
        summaries
        .Where(s => s.LRS.HasValue)
        .OrderBy(s => s.LRS)
        .Take((int)(len * 0.15));
Console.WriteLine();
Summary.Banner();
Console.WriteLine("TOP 15%");
foreach(var s in top15perc)
    s.Print();
Console.WriteLine();
Console.WriteLine("Bottom 15%");
foreach (var s in bottom15perc)
    s.Print();

Here is what we do when a request comes back with data:

static void downloadStringCompleted(object sender, DownloadStringCompletedEventArgs e) {
    var bigTuple =
             (Tuple<Tuple<string, string, string, string>, CountdownEvent, Summary[], int>)
              e.UserState;
    var tuple = bigTuple.Item1;
    var cevent = bigTuple.Item2;
    var summaries = bigTuple.Item3;
    var i = bigTuple.Item4;
    var ticker = tuple.Item1;
    var name = tuple.Item2;
    var asset = tuple.Item3;
    var subAsset = tuple.Item4;
    if (e.Error == null) {
        var adjustedPrices =
                e.Result
                .Split(new[] { 'n' })
                .Skip(1)
                .Select(l => l.Split(new[] { ',' }))
                .Where(l => l.Length == 7)
                .Select(v => new Event(DateTime.Parse(v[]), Double.Parse(v[6])));
        var timeSeries = new TimeSeries(ticker, name, asset, subAsset, adjustedPrices);
        summaries[i] = timeSeries.GetSummary();
        cevent.Signal();
        Console.Write("{0} ", ticker);
    }
    else {
        Console.WriteLine("[{0} ERROR] ", ticker);
        //Console.WriteLine(e.Error);
        summaries[i] = new Summary(ticker, name, "ERROR", "ERROR", , , , , , ,,);
        cevent.Signal();
    }
}

We ﬁrst unpack the Tuple we sent out originally, we then extract the Date and Price, create a Summary object and store it in the summaries array. It’s important to remember to Signal to the cevent in the error case as well because we want to print out the results even if some downloading failed.

And here is what you get for your effort:

SystemCodeAndData.zip

Comments

Barry Kelly

2009-09-23T08:36:49Z

Indeed. This is called momentum investing, and is probably at least partially responsible for the bubbles we’ve seen over the past few decades.
You can read more about it here - traders at Morgan Stanley described folks who used this approach as “fast money scum”:
http://marketprognosticator.blogspot.com/2009/07/real-vampire-squid-on-face-of-humanity.html

lucabol

2009-09-23T11:28:40Z

I’m afraid there is plenty of research on this factor (as there is on value). For a summary of it read here: http://www.cxoadvisory.com/blog/internal/blog-momentum-investing/Default.asp

ETFpickr

2009-09-26T21:37:15Z

FYI…
You can pull up a very similar set of data points for every ETF in the U.S. Market on my website http://ETFtable.com (also built in .NET)
You then sort by a number of different measures of performance and trend, and also ﬁlter the full list of ETFs to just a subset of ETFs that you track. Try it out and let me know what you think.

lucabol

2009-09-26T22:09:49Z

Lovely, I like the simplicity.
But I still like to be able to customize things even more. Then, consider my beautiful user interface :-)

mic

2009-09-27T09:20:55Z

take everything out and remember this statement “investment is not about timing it is about time”

lucabol

2009-09-27T11:07:22Z

Or was it the other way around? :-) :-)

jon

2009-09-27T14:31:55Z

Liked the new C# tricks. Thanks for sharing!

geo

2009-09-28T06:39:23Z

Pretty nice, but I would have gone with a scripting language for this job. I think the development speed would have been greater.

geoff

2009-10-17T20:23:16Z

@ geo re: Pretty nice, but I would have gone with a scripting language for this job. I think the development speed would have been greater.
Can someone please explain why a scripting language would make development speed better, and any other beneﬁts. I’ve heard statements like this and am curious as to why you would choose it over something like the author used, especially when he wrote it in a console without much overhead.

Tim

2009-10-19T14:43:46Z

Thanks for this great example on the power of C# .NET4. It has been a real eye opener for me. Amazing stuff!
It all makes sense except the Parallel For. I can see why it ends up working, but it seems less than optimal and or needlessly complex. Maybe I an missing something?
Parallel.For(0, len, i => {
var t = tickers[i];
var url = CreateUrl(t.Item1, start, end);
using (var webClient = new WebClient()) {
webClient.DownloadStringCompleted +=
new DownloadStringCompletedEventHandler(downloadStringCompleted);
webClient.DownloadStringAsync(new Uri(url), Tuple.Create(t, cevent, summaries, i));
}
}
);
Observation 1: Surely by calling DownloadStringAsync you are handling the work of actually doing the http get to a thread in a thread pool. Doesn’t this make the Parallel For redundant? The only thing that it is potentially parallelizing is the creation of the URL string which is pretty minor.
Observation 2: Surely you only need to create one WebClient object if you are going to use DownloadStringAsync as it returns immediately handing off the job to a thread in thread pool.
Observation 3: With a single WebClient you would only need a single DownloadStringCompletedEventHandler. Not a big overhead but still.
Observation 4: The using clause will call the dispose method on each WebClient. But presumably this will block until webClient.isBusy() is false. So these (potential) threads created by the Parallel For will create a url, hand off the work of actually getting the url to a thread pool (as a result of calling DownloadStringAsync) and then block on the dispose method until the get method. Seems a bit of an overkill.
I have not actually run you code so I am not able to try my theories out! But surely this is all you need?
<pre>
webClient.DownloadStringCompleted +=
new DownloadStringCompletedEventHandler(downloadStringCompleted);
using (var webClient = new WebClient()) {
for(i = 0; i < len, ++i) {
var t = tickers[i];
var url = CreateUrl(t.Item1, start, end);
webClient.DownloadStringAsync(new Uri(url),
Tuple.Create(t, cevent, summaries, i));
}
}
</pre>
Apoligies if the code formatting is shot!
Anyway thanks for a superb blog and apologies if I have misunderstood how WebClient.DownloadStringAsync works.

lucabol

2009-10-19T16:37:00Z

Hi Tim,
You are right. Parallel.For is unnecessary and I could have used just one WebClient.
Thanks for pointing this out.

lucabol

2009-10-20T09:37:59Z

I shouldn’t speak without trying out the code. Now I did and you are half right and half wrong. Parallel.For is somehow unnecessary as you are not doing much work in there, but multiple WebClients are needed.
Something along the line of your code gives:
Unhandled Exception: System.NotSupportedException: WebClient does not support concurrent I/O operations.
at System.Net.WebClient.ClearWebClientState()
at System.Net.WebClient.DownloadStringAsync(Uri address, Object userToken)
at ETFAnalyzer.Program.Main(String[] args) in C:ProjectsETFAnalyzerETFAnalyzerProgram.cs:line 238
I will change the code to use a simple for loop.

Tim

2009-10-20T12:24:43Z

I found an explication of why the Parallel For may be useful with WebClient.DownloadStringAsync here: http://www.informit.com/blogs/blog.aspx?uk=Is-this-really-asynchronous. Basically, DownloadStringAsync is not entirely asynchronous. It does the DNS lookup *on*the*calling*thread* ﬁrst. The comment at the end of the refereced url, explains why it might be still worth using a new thread to call each DownloadStringAsync even though that would appear to double the number of threads required. (There are separate IO and processing threads apparently)

Tags

Comments

Barry Kelly

lucabol

ETFpickr

lucabol

mic

lucabol

jon

geo

geoff

Tim

lucabol

lucabol

Tim