September 2017

[an error occurred while processing this directive]
(Click Message to Learn More)

The Wouda, Couda, and OODA of Building Performance Data Engineering

As part of the Project Haystack open-source community, I’m learning that small can equal big, and less can be more, and that ‘big data’ engineers and ‘building performance data engineers’ have important but different roles in the coming era of the IoT and machine learning.

Therese SullivanTherese Sullivan,
BuildingContext Ltd

Managing Editor, Haystack Connections

Contributing Editor

New Products
[an error occurred while processing this directive]
Site Search
[an error occurred while processing this directive]
Past Issues
[an error occurred while processing this directive]
[an error occurred while processing this directive]

What is a data engineer? According to Insight Data Science, a Palo Alto-based Institute, “a data engineer is a builder of tools and platforms to answer questions with data, using software engineering best practices, computer science fundamentals, core database principles, and recent advances in distributed systems.” In some cases, data engineers work with ‘big data.’ As part of the Project Haystack open-source community, I’m learning that small can equal big, and less can be more, and that ‘big data’ engineers and ‘building performance data engineers’ have important but different roles in the coming era of the IoT and machine learning.

Project Haystack has collected some of the best data engineers with building-performance experience in the world into its organization. The excitement of bringing these minds together was palpable last Spring at HaystackConnect 2017. “100% of us here believe in the value of data, and that is why we are here. But, in the broader market—across the 87 billion square-feet of real estate just in the US—we’re a small minority,” said Rob Murchinson of Intelligent Buildings.

While the numbers may be relatively small, the Haystack community is starting to have a big influence on the buildings industry. Looking around at the roomful of people that Murchinson was addressing, I started to take stock of the collective impact of their projects over the past six years — projects like top engineering university campuses, state-of-the-art data centers, Fortune 50 industrial campuses, major hotels in tourist centers, multi-tenant offices and retail complexes — all now Haystack-tagging enabled. They are making a dent in that 87 billion square-feet directly and indirectly as word of their successful analytics deployments travels fast. Offering an idea of the scale, Murchinson described one of his firm’s bigger customers that have deployed an analytics application that looks at a billion data points a month. That portfolio of buildings is geographically dispersed across the country, built in the range of 2 to 100 years ago, overseen by different facility management companies, and run by different engineering partners. “Without Haystack, tagging you cannot solve for this complexity,” he says.

The fact that the Haystack community, with no financial backing from academia or from the government, has solved the data mark-up language issue is huge. Big standards organizations and big universities have tried and are still trying. But, pragmatic solutions have yet to emerge from these efforts, and the buildings industry cannot wait for them to deliver. Haystack just keeps pushing ahead. Haystack is providing a workable semantics methodology for device data  that can be used to tag virtually any IoT or smart-and-connected device application today, and it keeps getting better..


It is validating to hear leading data scientists and researchers acknowledge the Haystack approach to semantic tagging. They like it for what it is, and, especially, for what it is not. From Milan Milenkovic of IoTSense:

“What is needed is an internet-inspired minimalist approach to semantic interoperability. The payload carried by a model should be simple descriptive data and meta-data annotation. The world doesn’t need another object-oriented data model. The annotation should be in a common format with recognizable names for what a thing is or how it is used—in other words, the Project-Haystack approach.”

‘Internet-inspired’ conveys concepts like linked-data principles, modularity, extensibility, readiness for decentralized applications, true data ownership for the people and organizations originating the data, and respect for their security and privacy.

Rob Murchinson teed up one more big-little paradox about data engineering when he said

“People passionate about data tend to want to make the results of their work as robust as possible as soon as possible. I would advocate not tackling the whole thing at once. Take little bites.”

He was aiming this advice at data analytics project teams. But, the same holds true for the Haystack organization itself. When you are an open-source organization with a diverse membership of various skill levels, domain experiences and different endgames, you need to be able to break up problems and workloads into pieces that represent a sensible division of labor for the collaborators. Murchinson is a skilled instructor of Agile software development methods, and he advises clients to work in terms of OODA loops—Observe, Orient, Decide Act.  He says:

“Get little pieces around that loop. Get the organization to start changing so that you can go faster and faster and faster. Data is a foundation toward an organization achieving their goals. We start by listening to our customers’ problems, which typically fall into three categories: sustainability, occupant experience and reducing operating costs. As you peel back the onion, it always gets back to data. But, the data alone doesn’t get you there. The bigger challenge is change management. Getting small problems through the OODA loop is how you will reveal your change management problems and be able to tackle them as well.”

[an error occurred while processing this directive]One small-bite increment employed by the Haystack organization is the concept of the Working Group. Eight new semantic tagging working groups were launched this summer. A team of Haystackers OODA-ed the tag set for air temperature in just a few weeks. At risk of belaboring the point about small things having great significance, who would have thought that a naming convention for ‘dry-bulb air temp’ was such a big deal? But looking over the names and organizations that jumped onto that working group, it is a hot topic of some consequence. Data engineers from one of the US National Labs, from a top commissioning firm out of California, an award-winning Master Systems Integrator out of the Great Lakes region, a leading energy engineering firm based in Salt Lake City, and others all offered their observations on the current tag set and recommendations for action. Their collective wisdom and experience will get folded into the Haystack standard tag set for weather points and be made available to all members of the community.

The other seven Working Groups are tackling some bigger topics, so their OODA loops will take some more time. But, new tag sets, tag dictionaries for specific building types, and constant improvement to the Haystack methodology is sure to result with this caliber of participant.

Per the definition in the first paragraph, this loosely organized community of volunteers using open source methods are doing data engineering. Project Haystack members share some common knowledge, skill and interests with 'big data' engineers, but a better title for them would be 'building performance' data engineers. They are domain-experts from their respective parts of the buildings industry, who see the value of the data created by smart devices and want to help the industry and society in general take maximum advantage of that value. The methodology they have created - Haystack tagging - is simple, lightweight, clean and easy-to-use and apply. That is what makes Project Haystack a huge accomplishment, and that is why it is gathering such momentum right now.


[an error occurred while processing this directive]
[Click Banner To Learn More]

[Home Page]  [The Automator]  [About]  [Subscribe ]  [Contact Us]


Want Ads

Our Sponsors