The Iterable Stream

Sometimes I find myself reusing certain paradigms in code in different projects. Many of these paradigms are the standard design patterns. There is one paradigm I’m using in a project right now that I’ve noticed I’ve used in other projects before. I call this pattern the Iterable Stream.

The Pattern

Say you have a data file or even just a stream that contains multiple records. A common example of this would be a character-delimited file which has records separated by newlines.

Python makes this embarrassingly easy with its ability to iterate over the lines of a file:

with open("path/to/some/file", "rb") as record_file:
    data_records = map(lambda line: DataRecord(line), record_file)
    for record in data_records:
        # Do something interesting with each record here.

In cases such as that described above, when using C#, I often expose the iterable records through a property:

public class SomeRecordSet
{
    private readonly StreamReader reader;

    public SomeRecordSet(StreamReader reader)
    {
        this.reader = reader;
    }

    public IEnumerable<DataRecord> Records
    {
        get
        {
            while (!this.reader.EndOfStream)
                yield return new DataRecord(this.reader.ReadLine());
        }
    }
}

This encapsulates the dirty work of producing records from the stream and allows you to do this:

using (var reader = new StreamReader(File.OpenRead("path/to/some/file")))
{
    var recordSet = new SomeRecordSet(reader);

    foreach (DataRecord record in recordSet.Records)
    {
        // Do something interesting with the record here.
    }
}

Though not as fancy as Python, using C#’s yield return syntax lets you easily make a stream iterable.

An Alternative

Alternatively, instead of exposing the iterable records through a property on SomeRecordSet, you could make SomeRecordSet itself implement IEnumerable<DataRecord>. Admittedly, this is a little more complicated, but not too much:

public class SomeRecordSet : IEnumerable<DataRecord>
{
    private readonly SomeRecordSetEnumerator enumerator;

    public SomeRecordSet(StreamReader reader)
    {
        this.enumerator = new SomeRecordSetEnumerator(reader);
    }

    public IEnumerator<DataRecord> GetEnumerator()
    {
        return this.enumerator;
    }

    IEnumerator IEnumerable.GetEnumerator()
    {
        return this.GetEnumerator();
    }

    private class SomeRecordSetEnumerator : IEnumerator<DataRecord>
    {
        private readonly StreamReader reader;

        public SomeRecordSetEnumerator(StreamReader reader)
        {
            this.reader = reader;
        }

        public void Dispose()
        {
        }

        public bool MoveNext()
        {
            string line = this.reader.ReadLine();

            if (line == null)
                return false;

            this.Current = new DataRecord(line);

            return true;
        }

        public void Reset()
        {
            throw new NotImplementedException();
        }

        public DataRecord Current { get; private set; }

        object IEnumerator.Current
        {
            get { return this.Current; }
        }
    }
}

Then you can just iterate over the records in the object, like this:

using (var reader = new StreamReader(File.OpenRead("path/to/some/file")))
{
    var recordSet = new SomeRecordSet(reader);

    foreach (DataRecord record in recordSet)
    {
        // Do something interesting with the record here.
    }
}

Conclusion

We’ve been using file streams here, but it should be clear that this pattern could be used for most any stream. Making use of C#’s yield return syntax or implementing the IEnumerable<T> interface can make a data stream iterable and hide the ugly details of doing so.

blog comments powered by Disqus