[Ibpsausa] DOE Job Task Analysis for Energy Modelers: Open for public comment until Nov. 28th

29 posts / 0 new
Last post

I would like to be grandfathered in please. Who is scrutinizing this
document (proposed regulation?) at ASHRAE headquarters? Have you had any
input from ASHRAE?

If you were to do a jta for President of the United States it might not be
so lengthy as the one for building energy modelers. (And, a lot of folks in
the current primary would be automatically excluded). Is it possible that
we have the cart before the horse? Perhaps we need a jta for building
architects and building design engineers of which energy modelers could be
a subset. I am sure that if DOE proposed a regulatory test procedure for
architects and engineers there would be plenty of comment.

I've looked at the document. What caught my attentions was the list of
"skills and abilities." These included such thing as "time management
skills," "critical thinking," and "spatial skills." Assuming that the
goal is to define *measurable* skills, we might want to give energy modeler
want-a-be s the SAT again.

Then I saw "common sense" and "patience" as skills and abilities. Now, not
withstanding the jta, I consider myself a competent modeler of buildings
and their energy systems. We don't need to vote on that but I also am
reasonably sure that "patience" has only recently kicked in as a personal
skill (maybe I am being optimistic). How are we going to measure the
"common sense" of someone who wants to apply to be an energy modeler?

Dr. Roth, it is not clear that very many in the building sciences field are
aware of your project, its history, and potential impact. Perhaps you could
get on the agenda at an upcoming ASHRAE meeting and explain the process and
your intent for the project outcomes.

Doug Hittle

Douglas Hittle's picture
Joined: 2011-10-02
Reputation: 0

Mr. Hittle,

My name is Daniel Studer and I am an engineer at the National Renewable Energy Laboratory. I also happen to be the project lead for DOE's commercial job/task analysis development work.

I can assure you that we have conducted extensive outreach to ensure that persons who operate in the spaces targeted by these JTAs are fully aware of the work being performed, including ASHRAE.

The energy modeler JTA itself was created over the course of three days by eleven practicing energy modelers who were guided through the process by a professional psychometrician. The names and associated organizations of each of these folks are listed at the back of the JTA, in case you are curious. DOE and NREL had zero input into the content of this document. And to ensure that the document is truly reflective of the industry, DOE has decided to make the documents available for public comment. That way, individuals such as yourself can provide DOE with constructive feedback to ensure that the document content is both appropriate and valid.

The intent of the project, as stated on the project website, is to:
Provide a basis for developing and comparing new and existing training programs in the commercial building sector. This will help individuals identify opportunities to enhance their professional skills, enable industry to identify an appropriately skilled workforce, and allow training providers to ensure that they are providing the highest quality product possible.

In short, we are trying to document the job as it currently exists so that training/certification providers can ensure that they are providing high quality products in line with industry's identified needs. Additionally, gathering this information using the objective approach that we have also has the side benefit of creating a sort of baseline document that can be used to help a training/certification end user better understand how their current skill set and existing training/certification options fit together.

I totally agree with you that items identified such as "time management skills" are not very measurable, and in fact seem both excessive and irrelevant, in the context of developing a training program or certification around this material. However, such information is very valuable in other contexts, which is why it is included in any JTA which utilizes the "developing a curriculum" (DACUM) method. The idea is that all aspects of the job should be documented so that the resultant analysis can be used for multiple purposes. For example, an individual wishing to develop a job posting may look at the JTA and include items such as "time management skills" or "spatial skills" in the job posting. They could then structure interview questions to better understand that person's abilities in those areas to make a more informed decision about a candidate.

In other JTAs that were developed under this project, SMEs also identified physical attributes necessary to perform the job (e.g., lift X lbs over head, see X feet). While these may also seem silly, they become very important when developing high risk assessments, such as licensure exams, in these spaces.

The real meat of each document is the DACUM chart located at the back of each. These charts identify the specific domain areas, tasks, and steps that the SMEs identified were necessary to perform the specified job. It is this content specifically that is of most use to training/certification providers, which is why DOE asked for comments on this specific chart in the Federal Register notice.

The proposed content blueprint tables (located near the front of each document), contain the SME's proposed weights for how often, and how important, each of the identified tasks is to the job. Such weights provide valuable context to training/certification providers by serving as guidelines for how much time should be devoted to each topic.

As part of this project, NREL will be facilitating a "survey validation" which will provide industry with the opportunity to adjust these weighting factors. However, to avoid the confusion that would occur with two "comment periods" occurring at once, this will not happen until DOE's public comment closes at the end of November.

Please feel free to contact me if you have any additional questions.


Studer, Daniel's picture
Joined: 2011-10-02
Reputation: 0

Dear All,

I took a brief look a while back and was uncertain what the Job Task
analysis was trying to do; it seemed very broad and full of "soft skills"
like common sense and patience, as mentioned by Doug. These are personal
attributes applicable to almost any job, any person and I was not sure what
they were doing in a job description. Kendra clarified this a bit in her
post of yesterday, but I find myself thinking that I still don't really know
what I am evaluating or why.

Kendra Tupper wrote earlier:

I am writing to encourage you participate in the public review of DOE's job
task analysis for energy modelers. Back in April, about 15 energy modeling
professionals (myself included) came up with this draft job description
(including required skills and knowledge) for energy modelers. This document
will provide the foundation for future education, training and certification
programs. It is likely that existing programs, such as the BEMP
certification, will need to show compliance with this.

For these reasons, I urge you to review and comment on this document! If you
think its sufficient and doesn't have gaps, that would also be helpful to
hear. The review period closes November 28, 2011.

James V. Dirkes II, P.E., BEMP , LEED AP

James V Dirkes II, PE's picture
Joined: 2011-10-02
Reputation: 203


Thank you so much for this clarification! I have to sympathize with Mr.
Hittle in that, despite the apparent efforts to "spread the word," I too
feel somewhat unsure of what the real implications will be for myself
and my colleagues in the energy modeling world.

My current impression is that the results of this study will ultimately
manifest as some sort of new licensure/certification/PDH requirements
for modeling services in future LEED and/or military/government
contracted work (i.e. DoD/GSA contracts)... does that sound right?

Maybe an illustration would be helpful... This is from the comment

"JTA is traditionally used by secondary and postsecondary educators,
test developers, and business, industry, government, and military
trainers to help identify core knowledge areas, critical work functions,
and skills that are common across a representative sampling of current

Can you provide an example of what JTA's have accomplished for those
working in the above industries?

For better or for worse, my concern is this study is going to add
additional layers of beauracracy to the job description... Great news
(enforced business) for those in the licensing/training industries I
suppose, but will there be a net benefit for the actual practitioners?
Are my concerns unrealistic or missing the point?

Thanks again for your time and efforts!


Nick-Caton's picture
Joined: 2011-09-30
Reputation: 805


Your concerns are entirely understandable, but perhaps a little overstated. The intention with this project was to produce documents that could be used by the corresponding sectors to improve existing, or if necessary, create new, training/certification materials, thus leading to a more qualified and effective workforce. If you take a look at the job categories we targeted, I think you will see that we focused on jobs that have a direct impact on commercial building energy use.

The development of specific training/certification materials however, is being left up to the training/certification providers in each industry. Our goal was to eliminate the heavy lift for such organizations by facilitating the identification (by industry, not by DOE or NREL) of what tasks were required in each job category, and what knowledge, skills, and abilities were required to perform each of those tasks. The idea is that organizations who have existing training/certification programs can self-compare their program to the developed JTA to determine if they are adequately meeting industry's needs. Or, for industries where few training/certification programs exist, organizations could use the developed JTAs to immediately start developing curricula (the DACUM method is often used to create curricula at the community college level). Through these mechanisms, the idea was to improve the quality and amount of training available to the end-user (in this case, the modeler), with the ultimate goal being a reduction in real energy use at the building level.

Additionally, the JTAs can be used as a standardized metric of sorts to evaluate existing training/certification programs. We are currently working on a tool that will allow an end-user in each of the six fields to examine what content different training/certification programs cover, using the JTAs as a baseline (so that folks are comparing apples to apples, instead of apples to oranges). That way, they can make a more informed decision about where to invest their time and money when pursuing professional development activities.

I do know that GSA is planning to utilize the JTAs to support efforts related to the Federal Buildings Personnel Training Act of 2010. What other items (new licensure/certification/PDH requirements for government/military contracted work, etc.) will come out of this work I cannot speak to, as those decisions are likely to be out of my control and should they arise, will likely be made within each corresponding organization, far from my purview.

I know this is another long response, and that I did not directly address several of your questions. However, I felt that clarifying the project purpose would actually eliminate the need for such answers. If you feel differently let me know and I'll try to go through and provide more specifics.


Studer, Daniel's picture
Joined: 2011-10-02
Reputation: 0


I'm afraid I can't agree that you conducted "extensive outreach" to prepare this document. For example, there was no discussion of this, before it happened, on any of the list servers before it came out. In addition, there was no "call for experts" that I was aware of. Normally, for a document like this that is sponsored by DOE, one would expect a call of some sort through the usual list servers.

In addition, I noticed from your list, that there is nobody who directly represents IBPSA-USA, nor even ASHRAE TC 4.7, which is the responsible ASHRAE TC for energy modeling. Although there are several individuals on the list who attend IBPSA and ASHRAE meetings and are well-known. It seems like little was done to notify others in IBPSA-USA or ASHRAE.

You should know that IBPSA-USA has also worked for some time on such an effort, which now exists as a wiki, under Joe Deringer's guidance. It also held a workshop in Boulder, and developed the BEM exam with ASHRAE. Yet, I see no reference to this or any of the previous efforts, nor efforts made to contact those of us who worked on this.

Finally, there are also at least a dozen or more educators within IBPSA-USA who teach Energy Modeling at the University level, yet not one of these individuals is mentioned in your list of contributors, only one academic appears on your list.

Hence, I suggest that if DOE is serious about making this a consensus document, then it needs to do a better job contacting the relevant organizations to obtain input and not rely on an email response after the fact.

Clearly, this is not the case with this document.

Jeff S. Haberl, Ph.D.,P.E., FASHRAE

Jeff Haberl2's picture
Joined: 2011-10-02
Reputation: 200


The following went out on the Bldg-sim listserve in March 2011.

Also, the IBPSA Wiki known as BEMBook is mostly empty. Maybe there's
another one that I am unaware of?

Jason Kirkpatrick

Opportunity Announcement

The National Renewable Energy Laboratory (NREL) is currently
sponsoring a project to develop (1) job task analyses (JTAs), which
identify and catalog all of the activities a worker performs in a
given job; and (2) the knowledge, skills, and abilities (KSAs), which
define the minimum requirements necessary for a person to adequately
perform those tasks, for the following six commercial building job

- Energy modeler
- Operating engineer/building technician
- Building energy auditor
- Energy/sustainability manager
- Commissioning/retro-
commissioning practitioner
- Facility manager

The project goal is to create "national guidelines" which will define
a common body of knowledge that any training organization will be able
to draw from when developing curriculum, helping to bring the core
competencies imparted by training organizations to their trainees into
common alignment. This body of knowledge will also be used by the
General Services Administration and the Department of Energy to help
meet the requirements of the Federal Buildings Personnel Training Act
of 2010.

To facilitate development of the JTAs/KSAs, Professional Testing,
Inc. is seeking current industry practitioners who are interested in
helping to define, promote energy efficiency in, and support their
field of practice by participating in these JTA/KSA development
workshops. Interested individuals are invited to submit their
credentials at http://proftesting.rapidinsites.com. Please note that
each JTA/KSA workshop will take place in Denver, Colorado and is
anticipated to last 3 full days (not including travel). Reimbursement
for travel costs up to a fixed amount and a travel per diem will be
awarded to individuals selected for participation. Additional project
details, including how practitioners will be selected and where to
direct project-related questions, can be found using the above link.

NREL and Professional Testing, Inc. are excited to present this unique
opportunity for individuals to have their voices heard in this
important discussion. Thank you for your time.

The NREL Commercial Workforce Development Team

Jason Kirkpatrick's picture
Joined: 2011-10-02
Reputation: 200


So much for me reading my email...in fact, I did indeed get this but did nothing.

However, even though there was an email about this, there was no discussion about this at the June IBPSA-USA meeting, nor was there a request for this to be put on the agenda at the June ASHRAE meeting for TC 4.7, Standard 140, or other committees of interest. In addition, Charles Eley has held several conference calls (some of which I've attended), which have discussed COMNet, yet this was not mentioned on any of the calls I attended.

In addition, such documents like this, if they are coordinated with ASHRAE, will usually be sent to RAC then to the TCs for comment, back to RAC and then back to DOE.

Finally, if this effort was truly important to DOE, why are there no other participants from the other National Labs? Why is NREL the only lab with expertise in this area? Or for that matter, why are there no individuals from outside the US? Or even Canada? Seems fishy.

Somehow, I still don't get a warm, friendly feeling about this.

Jeff S. Haberl, Ph.D.,P.E., FASHRAE

Jeff Haberl2's picture
Joined: 2011-10-02
Reputation: 200

My apologies Daniel, I suddenly realized that I am so old as to be before
your time. I was project manager on the BLAST energy analysis program and
my colleague and thesis adviser, Curt Pedersen was head of the BLAST
support office and one of the project managers of EnergyPlus. We both have
been very active in ASHRAE Technical Committee 4.7, Energy Calculations and
4.1 Load Calculation. I recently retired from Colorado State University
where I taught HVAC design including energy modeling (BLAST, EnergyPlus via
DesignBuilder, TRACE, TRANSIS, etc.). I feel fortunate that a number of our
former students are now your colleagues at NREL.

As you no doubt know, the words "certification" and "licensure exam" (see
your clarifying email) are loaded phrases in the consulting engineering
community. While most practitioners support professional engineering exams
and licensing, requirements for additional, new certification are not
likely to be widely embraced, at least not until they can be shown to have
some economic or other value. Furthermore, it may not be enough for
cognizant organizations to be aware of your JTA work. Groups like NCEES and
ASHRAE have to be early proponents of any certification or licensing
program that might result from your project. Otherwise Republicans and Tea
Partiers will claim that you are proposing more intrusive, job-killing
government regulations (be clear, *I* am NOT expressing this opinion).

Aside from the (probably) uncontroversial goals of guiding training and
education, and helping with job descriptions, what are the project goals
and expected outcomes. Are the results to be used to create an industry
standard (like standard 90)? Are parts of the work to be incorporated into
building codes? Do you hope to create a Professional Engineering Exam for
Energy Modeling? Perhaps the answer is "all of the above." The JTA report
is vague about this, perhaps intentionally.

Just exactly why are you and your team doing this work? Who are its
proponents? What are the hoped-for impacts? Who will be affected in the
long run? I am not trying to be critical by asking these questions but
without the answers it is hard to review the report.

Lastly, many energy modelers have a personal mantra that is not listed in
your report: "It helps to know what you are doing!" If I were hiring that
would be the main qualification.


Doug Hittle

PS Can we all assume that these email exchanges will be considered as
qualifying comments on the report as requested by Dr. Roth?

Douglas Hittle's picture
Joined: 2011-10-02
Reputation: 0


Apologies for the confusion. I had been responding previously only to the Bldg-sim listserv, as opposed to both the Bldg-sim and IBSPA-USA listservs. Below is my response earlier this evening to Nick Caton's email inquiry (again, sent only to the Bldg-sim listserv). Also, since things seemed to be quickly fragmenting, I have attempted to paste below all emails on this topic, so that everyone can easily reference them. I will follow up to the two most recent emails in a separate posting so that others may follow the conversation more easily.


Previous email sent to the Bldg-sim listerv only:

Studer, Daniel's picture
Joined: 2011-10-02
Reputation: 0

Please note that this email contains responses to postings by both Mr. Haberl and Mr. Hittle.

Mr. Haberl:

The purpose of each JTA is to capture the job as currently practiced. Such a baseline document is essential in order to fairly and objectively identify knowledge and training gaps that exist between where any industry is and where it "should" be.

As such, we went to extreme efforts ensure that psychometrically valid sets of SMEs were selected to define the documents. Besides having to prove that they were current practitioners in the field, applicants were screened by a psychometrician (not DOE or NREL) using criteria such as geographic location, years of experience, and industry credentials to ensure that the selected group was broad with respect to all of these criteria (this is outlined in slightly more detail in the JTAs themselves). Bias in any one of these areas (experienced folks over novices, people from the east coast vs. the southwest, etc.) would have favored a particular group and skewed the resulting document in favor of that particular group's thinking. Since we were attempting to adequately capture the job as practiced in the U.S. (not as practiced by experts in the U.S., or by academics in the U.S., or how it should be done according to organization X, etc.), we wanted the broadest U.S. representation possible. This is also why there were no international participants.

Clearly, the inclusion of any professional society or organization directly involved in the development of such a document would also skew, whether meaning to or not, the resulting document toward the viewpoint of said organization. Indeed, to prevent this from happening, no outside reference materials were allowed into the workshops. Thus, the material presented in the JTAs comes directly from the first-hand knowledge of the participants. The resulting JTAs were all created by practitioners detailing what it was they did in their jobs.

I believe that the Boulder meeting you mentioned was the BEM Innovation Summit, which was hosted by RMI in Boulder last spring. The following is from PDF page 36 (document page 32) of the post summit report (http://rmi.org/Content/Files/BEM_Report_FINAL.pdf):
"Additionally, the DOE is currently sponsoring a project to develop (1) job task analyses (JTAs), which identify and catalog all of the activities a worker performs in a given job; and (2) the knowledge, skills, and abilities (KSAs), which define the minimum requirements necessary to adequately perform those tasks, for six building job categories, including "energy modeler". The project goal is to create "national guidelines" which will define a common body of knowledge that any training organization will be able to draw from when developing curriculum, helping to ensure consistent core competencies among training programs."

An action item identified in that same report (PDF page 38) reads: "Respond to Doe's RFP to develop "national guidelines" for energy modeling education."

Mr. Hittle,

Congratulations on your recent retirement; that is one of my near-term goals (to be accompanied by a private island though).

I believe that you may not have seen my previous email to Nick Caton, which I believe (again, opinion) may answer most of the questions you pose. If not, please feel free to let me know and I will do my best to answer them. Like I said in my last email, this thread is getting a little fragmented.

However, I can answer one of the questions you pose here. The Federal Register notice released by DOE (http://www.gpo.gov/fdsys/pkg/FR-2011-10-14/pdf/2011-26645.pdf) details the means by which they will accept public comment. While email is one of those means, said email must be sent to a specific email address and accompanied by a signed copy (there are other, less burdensome ways to submit comments as well). So in response to your question, no, this correspondence will not be officially documented as comments received on the documents.

Thanks all for the lively discussion!


Studer, Daniel's picture
Joined: 2011-10-02
Reputation: 0

Thanks for your responses Daniel. I now understand that your (DoE's)
purposes are to provide guidance to educators, trainers and job description
writers and to assist GSA and DoD in developing personnel training
programs. I gleaned this understanding from your remarks and from the
paragraphs from DoE solicitations and emails that you quoted. This being
the case, a lot of unproductive pot-stirring can be avoided if you were to
simply broadcast the fact that neither NREL nor DoE intends for the JTA
document to be the basis for any standard, building code, certification
granting process, licensing process or any other form of regulation. When
you do this, practitioners will be able to review the document in its
proper context and they will know that it won't and is not intended to
infringe on their right to practice.

Also I hope you will consider all input no matter how it is transmitted and

Until tomorrow.
Doug Hittle

Douglas Hittle's picture
Joined: 2011-10-02
Reputation: 0


I still don't agree with your process. Your logic is flawed and most likely so is the document.

You excluded those organizations that could/will benefit from your document for no valid reason, and you have excluded known educators in this field, experts in the field, and other national labs.

For the most part this was developed off-line, out of public review. Hence the results of any such process cannot be trusted and the document must be regarded as being second rate.

Jeff S. Haberl, Ph.D.,P.E., FASHRAE

Jeff Haberl2's picture
Joined: 2011-10-02
Reputation: 200


I was reading all of these emails this morning and I will admit, I've
only scanned the Job Tasks Analysis document. But I think we should all
put this in the perspective that this is a first step in Energy Modeling
becoming a profession on its own. If I understand this correctly, these
classifications are used as standard job definitions for things like the
Occupational Outlook Handbook http://www.bls.gov/oco/, and to gather
statistics on energy modeling as a profession, like how many are there
in the country, what is their typical education, salaries for different
levels of experience.

I am an energy modeler, much more than an architect, or a mechanical
engineer. I for one am excited that energy modeler is becoming a
stand-alone profession, rather than just a offshoot of one of the allied

There may be flaws in the definition, and the process, but this kind of
marks a coming of age of our field, and I think that is kind of

Chris Baker


Chris Baker's picture
Joined: 2011-10-02
Reputation: 0

Agreed! The term "Energy Modeler" at this point has always been loosey
goosey and has only recently become more popular due to LEED (as much as
I hate to give USGBC credit haha)

Creating a standard "position" will be good long term to develop more
awareness of what skills we bring. In fact, I'd rather us be called
"Building Analyst" rather than Energy Modeler. Our skills and job go
beyond just "modeling", I think. We help analyze buildings whether for
energy, costs, etc.! I see project managers bringing together and
leading the resource side of things and a "Building Analyst" bringing
together the technical side of things (helping analyze architect's
decision on glazing, walls or mech engineers HVAC systems, etc. etc.)

Hope the morning writeup wasn't too confusing, having had a cup of
coffee yet!!

William Mak, LEED AP BD+C

Will Mak's picture
Joined: 2011-09-30
Reputation: 200

I thought BEMP was the first step to make energy modeling a profession of
its own? Hasn't ASHRAE already analyzed the required skill set, work
experience, and education when developing the test? I work with several
large incentive programs in north east, and was planning to push for making
BEMP mandatory for modelers working in the programs. DOE Job Task Analysis
project will weaken my chances of succeeding, because I can see
administrators delaying the decision in expectation that some alternative,
better certification will come out. Modelers may defer taking BEMP for this
same reasons. If DOE document has to be released, I think it is very
important that it acknowledges the related prior work, and clearly defines
where this parallel effort fits in.

Maria Karpman

Maria Karpman's picture
Joined: 2011-09-30
Reputation: 0

I don?t think there is need for ?sponsoring a project to develop (1) job task analyses (JTAs), which identify and catalog all of the activities a worker performs in a given job; and (2) the knowledge, skills, and abilities (KSAs), which define the minimum requirements necessary for a person to adequately perform those tasks, for the following six commercial building job categories: - Energy modeler - Operating engineer/building technician - Building energy auditor - Energy/sustainability manager - Commissioning/retro- commissioning practitioner - Facility manager?.

A JTA is not required for US President. Energy analyst is not equivalent to a surgeon. Life is not at stake. But there is a need for such a document for education and certification.

At large A-E design firms the task of using energy programs is usually assigned to mechanical engineers who have recently graduated because engineering theory is fresh in their minds. The programs are based on physics and engineering theory. A 1 or 2 year graduate program would save the A-E design firm of the expense learning this subject on the job.

However a program like eQUEST can be used by someone with little knowledge of theory. It is like driving a car without knowing the theory of internal combustion engine. The analogy should be the car designer and not the driver. Education (not 3 day seminars but 2 year graduate programs) and certification are important but so also is design experience to perform the energy analysis within a full understanding of the building life-cycle process.

Why is there so much emphasis on building energy modeling and simulation? The importance of this work is exaggerated.

The results of energy programs are not used in design and therefore not used in construction and operation. Energy program results are used to compare alternative design options and make decisions when selecting envelope, systems and plant. They are also used to show energy code compliance and for sustainable buildings certification.

A-E design affects the building products & equipment selected, construction and operation. Design decisions are also based on * first & maintenance costs; * reliability & durability of systems, plant and equipment; * ease of operation and maintenance; * availability of parts & maintenance staff at the location; * environmental impacts.

Given a set of drawings, schedules and specifications, two expert users of the same energy program will come up with different results. The same person using different energy programs will have different results. The results of energy programs for energy code compliance and for sustainable buildings certification are rarely checked with the completed building performance in operation. I cannot find this information for any energy program. There is also no information of which energy program was used for a LEED certified building and for all buildings that now have to comply with energy codes.

California and Florida had energy codes before the year 2000. Most other States introduced energy codes after 2000. 10CFR434 is not stringent. Computer energy analysis before the year 2000 was rarely done but the buildings were designed for efficient energy use by reputable A-E design firms. ASHRAE Std. 90.1 describes standard existing practice (except the 40% windows) by A-E design firms.

One million sqft office bldg (40 stories, 25,000 sqft/floor) in a large city downtown (including the land) costs $250 million at $250/sqft. The A-E design fee is 3% or $7.5 million. At an average A-E labor cost of $100/hour (including overheads) the design time is 75,000 hours. Energy analysis of an office bldg using eQUEST or TRACE takes about 100 hours (2.5 weeks at 40 hours/week) or 0.13 % of A-E design time or 0.004% of total building cost.

The importance of building energy analysis is inflated.

Varkie Thomas

Varkie Thomas's picture
Joined: 2011-09-30
Reputation: 0

My experience is different than yours Varkie. We often need to simulate buildings to a high level of accuracy (too high in my opinion) in order to obtain state and federal rebates and incentives. An entry level designer will most likely not be able to hit those targets and I would even say most staff level engineers would have a difficult time. It takes some senior level engineering to understand how the building systems operate; what is a normal result; and most importantly, how the program is going to react when the inputs are changed.

I guess if the output reports are not reviewed with any scrutiny and the targets have a wide range of acceptance, you could hand it off to an entry level engineer. But hitting an electric and natural gas utility bill, month-to-month and a yearly overall, simultaneously within +/- 10% accuracy will take a senior level understanding. Also, it?s not the time that is important as much as the ability to do it. The cost to my company for my time to perform the simulation is irrelevant compared to the rebates and incentives that will be unavailable, and as a consequence the construction project that won?t move forward, if the simulation can?t be performed.


Michael A. Eustice's picture
Joined: 2011-10-02
Reputation: 0

Michael: I am retired and I think like someone who lived in days when there was no LEED certification and no energy code compliance. Since the first ASHRAE Std.90 came out, most if not all commercial buildings in the US were designed for energy efficieny. The last sentence should have read "The importance of building energy analysis appears to be inflated relative to the total building design because of the amount attention given to it by lawmakers". Example - DOE Job Task Analysis for Energy Modelers. Energy efficient buiding design is important both in education and practice but it is just one small subject in the A-E design process. I am now a Research Professor with CoA Ph.D. program at IIT specializing in this minor topic. Varkie

Varkie Thomas's picture
Joined: 2011-09-30
Reputation: 0

I may be stepping out of line here but aren?t we in this energy mess because buildings weren?t design for energy efficiency at all due to extremely low fuel costs back in the day?? Isn?t one of the main points of modeling to justify ROI investments as opposed to the cheapest first cost systems by owners?

We finally have some documentation process to justify to owners/clients that hey, you can?t just keep putting the cheapest systems, it?ll cost more long term!!! Are you OK with that? Or do you want to look at other options with this data analysis I?ve done??

?Michael: I am retired and I think like someone who lived in days when there was no LEED certification and no energy code compliance. Since the first ASHRAE Std.90 came out, most if not all commercial buildings in the US were designed for energy efficieny. The last sentence should have read "The importance of building energy analysis appears to be inflated relative to the total building design because of the amount attention given to it by lawmakers". Example - DOE Job Task Analysis for Energy Modelers. Energy efficient buiding design is important both in education and practice but it is just one small subject in the A-E design process. I am now a Research Professor with CoA Ph.D. program at IIT specializing in this minor topic. Varkie?

William Mak, LEED AP BD+C

Will Mak's picture
Joined: 2011-09-30
Reputation: 200

I've been following this discussion closely and want to add a few things:

1. The value of energy modeling has so far not been realized due to the
emphasis of using modeling on the back-end for LEED points and incentives,
instead of on the front end for design decision making. As long as we are
arguing about the 'importance' of energy modeling, we need to recognize
this fact.

2. Any time a new job description, exam or set of prerequisites is put
forth, many energy modelers get defensive or nervous because they might be
a very capable modeler, but their particular educational background or set
of skills does not fit into the current box that is drawn. For example,
engineers often are the ones completing energy models because on any given
project team they usually have the best understanding of how the HVAC
systems in the building work. As a result, I often see "licensed PE"
associated with energy modelers. MEP professionals and energy modelers
share many skills, including understanding of HVAC systems and controls
systems and a fundamental knowledge of thermodynamics. However, there are
many skills MEP professionals need that energy modelers do not need. MEP
professionals also need to know how to size ductwork, select appropriate
components, create a set of drawings, write specifications and complete a
punch list.

Likewise with architects, many skills overlap between their discipline and
energy modeling, but energy modelers don't need to know how to size
structural elements or create a rendering.

In addition, there are many skills that are not inherent in either of the
two above professions, that are required for energy modeling. Most
important is an in-depth knowledge of the available computer programs,
including their abilities and limitations. I have also found that a good
understanding of computer programming has been invaluable as a way to
respect and use the software for it's strengths.


No Username provide's picture
Joined: 2011-09-30
Reputation: 200


Thank you for your interest in this project. To allow DOE to consider your comments, please submit them in accordance with the methods detailed in the Federal Register notice, available at: http://www.gpo.gov/fdsys/pkg/FR-2011-10-14/pdf/2011-26645.pdf.


Studer, Daniel's picture
Joined: 2011-10-02
Reputation: 0

Apologies for the following wall of text? those recognizing me from the lists know this is a tendency of mine =)!

This discussion is beginning to serve as a great example for why a formal assessment/clarification of the job would be requested! Following are some thoughts that may serve as further input for the study:

?Energy Modeling? is a service that means many things to many people. These lists include a large body of highly intelligent and competent practitioners. Those who have earned my respect collectively have a very diverse background of education levels, design experience, day-to-day responsibilities, and often unique expertise/perspectives within the realm of energy modeling.

Contrary to Varkie (who I hold in very high regard), I can assert my modeling efforts are most often engaged for the primary purpose of informing and improving design, across the trades. Compliance/LEED modeling comes in at a close second within my experience pool. The importance of this skillset is most recognizable by the satisfaction of the design team members and owners who are constantly learning new things and seeing their buildings designed in a more cohesive & energy-informed fashion, not by a LEED plaque. I believe today?s design teams and owners (including the government) are incrementally learning and recognizing the potential value a competent energy modeler brings to the table during the building design process, and it is showing in the language used for contracting new work. LEED as a whole deserves credit as a catalyst for bringing this to light. As more and more quality modelers join our ranks and blow away owners and design teams with quality work, I think we can expect the demand to rise, and the need to differentiate who out there is a quality modeler will become more prominent. I see this JTA study (and the legislation initiating it) as a natural progression by the powers-that-be to approach this challenge in a very deliberate fashion. Defining the job would be a necessary early step along this path.

The best modelers (and to be clear I merely aspire to join this category) each have differing ideas regarding what represents an ?appropriate? degree of effort (hours for a given model), what constitutes and ?acceptable? degree of modeled accuracy and what value this skillset/service really represents. I think any study trying to nail down those particular issues by asking the modelers would come up short. The answers vary per-project and are ultimately answered by those paying the bills.

My experience is almost certainly different than both Michael and Varkie in terms of what degree of accuracy/time-investment is required on my ?average? project. Select past discussions on the lists have helped me to understand that I personally spend a relatively high number of hours on any given project, but also have some more stringent expectations for what my models represent. This is partially the result of my personal abilities, experience and standards of quality, however a large driver is the clients & fellow consultants who I work with and the relative complexity of their associated projects.

This discussion is brushing up against a very delicate topic requiring nuanced language? I want to make some positions of mine crystal-clear for those soliciting energy modeling services today, those seeking input for this JTA study, and those who may use (or mis-use) this study?s results:

1. One cannot measure or judge the quality of an energy modeler or his/her work by the amount of time they spend on a project or any given part of a project. What is appropriate for one project and modeler will be different for the next. I expect modelers with less experience than me to produce high quality work, but to require more time to do so. On the flip side, more experienced modelers should be capable of producing an equal quality of work in less time.

2. Neither can you rely on the how close a predictive model matches reality (utility bills) to evaluate a modeler. Calibration modeling for existing/renovation work is a particular and distinct skill set within energy modeling, for which this metric has limited application? but predictive modeling is far too often mischaracterized and mis-used at a means of knowing the unknowable.

3. Modelers, when permitted (and compensated) to be actively engaged in the design process, are to be regarded as a full and distinct member of the design team. Our industry and even our modelers often undervalue the potential for these services. I hope the weight and responsibilities of this ?active? role is conveyed in the JTA. My current sense is someone might on first reading this come away with the impression energy modelers are only ever periphery to the design process, and someone may not realize the potential impact a modeler can have as an integral part of the design process. To be certain, both roles exist for today?s modelers, and I expect this trend to continue.

4. I want to echo/reinforce Maria?s concerns. I think it is logical that those referencing this study in the future would be well served to be made aware of the present BEMP certification, who is behind it, and what it entails relative to the definition of what an energy modeler is. I recognize there?s an aversion to referencing or drawing from any industry groups? but ignoring them entirely makes the task of defining the trade all the more difficult, at best.

5. I?ve said a lot regarding metrics you cannot effectively use to identify a quality modeler ? and I realize that is only so helpful. Perhaps none of this is considered appropriate for defining the job?? Still, if I were interviewing to hire a modeler, prioritizing quality in the end result and pushing speed/budget concerns to the side, I think a good start would be to compare how willing/comfortable the modeler is to discuss and share their modeling decisions. Put another way, how ?open book? are the calculations?


PS: I propose whoever is in charge of collecting input for the JTA simply submit a copy of the discussion threads as they branch about and resolve themselves here on the lists. Any single post or contribution taken out of context of the ongoing discussion might be misunderstood otherwise.

Nick-Caton's picture
Joined: 2011-09-30
Reputation: 805

Hey Will,

I posed similar questions to an individual who started his career in mechanical systems consulting well before I was born. His perspective is that energy-conscious building design has come into vogue and receded in waves over time, and will probably continue to do so as the availability of our energy resources become varies in the future. The difference in this wave, and the reason it?s such a big deal, is that now we have all these fancy computers and software than can answer questions and reveal trends that could not be broached in the past.

I personally think it?s a stretch to suggest energy efficiency was not considered at all in the past few decades. That may be true of some, but when 90.1 and energy code AHJ?s came along demanding less than 1.1 LPD for office lighting and using economizers where they make sense for packaged equipment and insulting the hot water lines? many of us simply shrugged our shoulders ? it was already standard practice.

I don?t believe anyone on these lists is presently arguing against any of your other points ;).


Nick-Caton's picture
Joined: 2011-09-30
Reputation: 805

Ok. One last comment and then I have to get back to paying work.

Listed below are pretty much all of the skills and abilities listed in the
DACUM in the DoE report:

? Computer skills ? Critical thinking skills ? Logical thinking ? Time
management? Engineering skills ? Math skills?Analytical skills? Research
skills? Organizational skills ? Physics? Creative? Logical thinking?
Observation skills? Safety conscious? Project management skills? Analytical
skills? Patience ? Persistence? Ability to interpret drawings.

In most Universities, ours included, we strive to inculcate all of the
items in the list into every student who graduates in any scientific
discipline. Our best engineering students graduate with these skills. Ergo,
they have the skills they need to be energy modelers. How is repeatedly
listing these abilities going to help anyone?

In the "Specialized Knowledge" column there is some unevenness. Knowledge
of the "Building design and construction" is all encompassing while
knowledge of "Vertical transportation" is limited and it is not clear what
knowledge is required for energy analysis.

Have we now ALL agreed the the DoE document will not be used for
certification, standards, licensing, etc.? YES or NO!!!

Doug Hittle

---------- Forwarded message ----------

Douglas Hittle's picture
Joined: 2011-10-02
Reputation: 0

Hi Doug,

I think the ?Skills, Abilities and Attributes? section and columns in the DACUM may be better coined ?personality traits? in layman?s terms, from the preceding section. I?m guessing they?re intended for something like an aptitude test for high school kids?

I?m not sure anyone is asking whether this report/effort is to be scrapped entirely ? for context I vaguely recall reading about legislation being passed mandating this sort of study be performed? Here?s a link I found in my previous emails (thanks Danny!): Federal Buildings Personnel Training Act of 2010 .


That said, Doug?s concerns below resonate further with an uncertainty I think many of us share over how this report may be used, or more specifically mis-used. The document leads off with discussing what a JTA is and what it?s for, in relatively vague/generic terms (not specific to energy modeling). You can pretty well tell from our community?s responses that this section needs to be fleshed out with more specific examples of who is asking for this study and why, and to better define the intent/purpose/projected end uses of the study. This added context will better present the collected data and encourage more constructive feedback.

Some additional constructive thoughts along the lines of improving what?s there, based on my current understanding:

? I see a lot of potential mis-use/misunderstanding of the ?weighting? presented for various job tasks, from an educational/training perspective. Does this represent the time-distribution of an experienced modeler? A new modeler? For what sort of project (LEED, retrofit consulting, schematic analyses)? I can confidently say this weighting distribution can and does vary widely between projects depending on the circumstances and who is involved, and for any given modeler. Some discussion of what these percentages specifically represent is in order lest they be misused by the wide variety of sorts in the target audience (educators developing curricula for college students vs. professionals, or government contract writers thinking of experienced modelers, etc).

? There are a number of items tabulated under ?Skills, Abilities and Attributes? which are more appropriately situated under ?Specialized knowledge,? such as ?Ability to interpret weather file structure.? Likewise, I don?t believe ?Geometry? or ?Safety? constitute ?specialized knowledge? as the report currently describes it.

? While I am able to understand what is being referenced for the various attributes/knowledge? someone outside or periphery to our industry would most definitely not. More than a few of these terms are either too broad/vague or too technical/specific to assume most would fully understand what?s being referenced by the panelists. Such entries warrant further definition.

? Section 8 reads: ?The panelists concluded that a practitioner must master the knowledge in both tables to be competent as a commercial building energy modeler.?

o Reading and applying this literally, it would appear the panelists have concluded anyone who has not mastered ?Computational Fluid Dynamics? (among other things) is an incompetent modeler.

o Based on my understanding of some of the panelists? backgrounds, I would be very surprised to find this is a true consensus, or that this specific wording has the full endorsement of everyone involved.

o Words like ?competent? and ?mastered? are dangerous if used too liberally in this sort of study, considering the possible usage of this report as we currently understand it. If you intend to use such terminology, carefully consider whether you are setting up a doomsday scenario for the industry, where the newest generation of modelers will be unable to practice/learn energy modeling because they are not ?masters? of such a broad range of knowledge.

I suppose this helps bring up another concern. Where those participating and conducting this study may be totally rational and intelligent in the industry and subject matter at hand, this study is in part intended for use by those outside of our industry (HR staffing, politicians) who may not use its contents in a rational fashion. Context (again) that we probably take for granted would seem a necessity to prevent such mis-use.

Hope this helps everyone involved!


Nick-Caton's picture
Joined: 2011-09-30
Reputation: 805


While I agree with some of the concerns regarding the Skills, Abilities and
Attributes section of the document, I think that people are overlooking the
real substance of the document, namely, the Duties, Tasks, and Steps that
were outlined to define the energy modeling process.

I would like to hear some specific comments regarding that portion of the


Jason Kirkpatrick's picture
Joined: 2011-10-02
Reputation: 200

Dear All,

As an emerging modeler I was excited by this discussion.

I found it quite exceptional, recognizing among participants highly
respected professionals whom I learned recently from this discussion group.

As a member of modeling community I consider this email list as a great
source of real-life professional information.

Having said that I'm wondering why the value created by respected
professionals can be wasted for the sake of an instruction. Why
the government cannot use the comments that has been already expressed and
recorded, despite the fact that it wants to know these comments.

Moreover, email-service allows to save all posts in a single pdf
preserving the chronological
order of Qs and As, which otherwise would be very time consuming to

Being thrilled by this discussion i decided to do a simple thing as to
attach a pdf, comprising last 27 posts of this discussion.

(Surprisingly, the pdf-file made by Mac is 10 times smaller than the one
made by Windows)

Please forgive me if I did violate some US public/government relation
orders, since I'm from another country.

Best Regards to everyone,

Pavel Dybskiy

Pavel's picture
Joined: 2011-10-18
Reputation: 0

Jason et all:

I will start with some specific commentary, then step back to discuss
the overall scope of this JTA and my opinion of how the broader goals
should be refined. I'll preface that this collection of feedback is
blending together discussions I've had recently and in the past with
other colleagues on and off the lists over the topic - these thoughts
may be considered reviewed and coming from more than one of us, in
short. I hope this helps improve and refine the current efforts to a
positive effect for everyone involved.

The document needs to be reviewed for sensible and clear terminology
usage, and to remove any redundant items (stuff a spell-check is not
going to pick up). For example, Table 7.A.6.a&b lists the two following
tasks side-by-side:

* Review baselines required for Codes, standards, and protocols

* Identify additional baselines

The distinction is not clear from the provided context. On my first
reading I mentally corrected both to reference "baseline requirements,"
but that doesn't read particularly well if these are supposed to be two
distinct tasks either... Baselines do not always come from a
standard/protocol, as I'm to discuss more thoroughly further down...
perhaps that's what's intended?

Specific to the gathered tasks and workflow presented, something I
touched on earlier was how the entire report seems structured around a
specific modeling role and scope of work. Reviewing what is presented,
this scope initially appears to have a few identifiable characteristics:

* The energy modeling consultant is working periphery to the
design process (not a member of the "design team")

* An emphasis appears placed upon identifying and constructing a
baseline model per existing conditions, suggesting the scope of work is
for a renovation/retrofit project.

* While the modeler is supposed to be familiar with LEED and
similar standards, the duties/tasks/steps for the project at hand are
not specific as to whether a formal performance rating method is being
followed or documented. If that is the case, I would expect at least
different weightings between the relatively tasks presented.

It has since been clarified to me by one of the participating panelists
that the above perceived set of scope characteristics are likely the
combined result of:

* Discouraging panelists from referencing any specific standard
or protocol (i.e. LEED), where such compliance documentation & review
responses can at times represent a significant portion of the time spent
on a modeling project.

* Panelists were asked to iterate every task involved in both
new construction and existing building projects, which introduces
communicative and information-gathering tasks that would not apply under
different scope/project circumstances.

These observations lead me to point out and explain something more
fundamental to energy modeling that the rigid structure of the JTA study
procedure/format seems naturally prone to miss. As has been repeatedly
noted, "energy modeling" is a broad term that describes a wide breadth
and depth of services, knowledge and skillsets. I would not object to
the duties currently presented being adequately inclusive for the
workflow of a very specific scope of work, but the box as drawn both
misses and diminishes other areas of modeling that I think are integral
to the job description and the industry... If the job of "energy
modeler" must be defined in a "workflow" fashion (duties/tasks/steps) as
currently presented, I would instead propose coming up with a workflow
to describe three separate and distinct categories of modeling services:

* "Compliance Modeling" - Using energy models to demonstrate
code/standard compliance and using prescriptive procedures such as the
Performance Rating Method to demonstrate relative performance against an
established design standard (i.e. LEED, Title24, EPAcT)

* "Renovation/Retrofit Modeling" - Modeling inclusive of
existing buildings and systems, characteristically involving additional
measurements and information gathering to define and calibrate baseline
models tied more closely to reality, with the explicit intent to make
informed cost-benefit and ROI analyses and recommendations for cost
saving measures.

* "Design Analysis Modeling" - SD/DD early stage decision-making
modeling, wherein baselines and results are built on relatives and
isolating the variables in play to analyze and inform design decisions.
Baselines are dynamic to suit any given study, and inherently cannot
follow a prescriptive modeling standard assuming a complete design.
More description to follow.

The order and weighting of the JTA duties/tasks/steps descriptions, as
well as the areas of critical knowledge, varies for each of the above
categories. For example, one doing "Renovation/Retrofit" modeling would
generally want to first invest a good deal of time building a baseline
model to match the properties and behavior of the existing building and
its contents. An adequately calibrated baseline would functionally
serve as a tool useful to identify the relative impact of individual and
combined cost saving measures, and might also inform parallel ROI
analyses for the building owner's benefit. On the other hand, if one is
being asked to provide modeling services specifically for LEED or
another prescriptive performance rating, then he or she would generally
be best served to start by accurately modeling the proposed design, from
which the baseline can be constructed. In either instance, the steps
and questions to be answered for "information gathering" stages, as well
as the order and weighting of each "phase" varies quite dramatically.

Note: The three above categories of modeling are not mutually exclusive
- any single project could conceivably incorporate any combination of
the three, but they each draw on distinct skill sets. Obtaining the
experience & knowledge to be an "expert" (or "qualified" or "competent"
... all potentially loaded terms I would advise using judiciously) in
any one sub-field of energy modeling does not make one a master of all
three areas, and simultaneously having little experience in any one area
does not render a modeler incompetent either.

Much can be said to further differentiate and define these 3 areas, but
where the first two are I think at least immediately
identifiable/distinguishable to those within and periphery to our
industry, I'll provide some extra attention to help define "Design
Analysis" modeling for this discussion - which I consider to be the
least well-defined within our present industry (discussions surrounding
the next iteration of LEED suggest this may change in years to come).
Design decisions for new construction are often made without a finished
proposed design to reference nor an existing building with energy bills
to calibrate to. "Baselines" instead must be built in direct response
to the queries posed, based on what information is and is not available.
To illustrate with an example I've used previously: one may wish to pose
a question like "What is the ideal WWR for the walls along our Southern
facing building facades, considering the savings of daylighting controls
and the impact of the resulting envelope thermal losses/gains?" This
sort of question can be answered by a competent energy modeler, but to
answer this question in a time-efficient fashion one has to have
distinct skills of being able to identify, isolate and be ready to
communicate & discuss:

(1) the measures being explored,

(2) the variables which affect the measured outcome, alongside
assumptions of the design that must be made, and distinctly

(3) the parts of the building which do not have an appreciable impact on
the result.

This last point is given a bold font in particular because in my
experience, "Design Analysis" modeling occurs during the more fluid
stages of design which demand producing feedback at a relatively fast
pace, and the ability to holistically recognize what to focus on and
what not to spend hours defining is critical to that end. I wager there
is a certain combination of both energy modeling experience and building
design experience that someone adept with "compliance modeling" would
not need, but really facilitates providing this sort of quick,
informative and accurate "design analysis" feedback.

In light of the proposed categories of modeling services, let's step
back to the topic of what the broader term "energy modeler" means in
relation to this study:

If the entire JTA form & structure is built around defining and
weighting how much of any one "task" is performed for everything that
falls under the job description of "energy modeling"... I find it hard
to imagine such a broad single distribution could be usefully applied
for a specific job or qualification description. The JTA approach to
defining the job will likely be more constructive and useful if distinct
sets of "job blueprints" are developed for each of the three categories
described above.

I recognize this is a wall of text, but this JTA study presently seems
too narrow to claim "identified tasks and weighting factors accurately
represent the job of a commercial building energy modeler." Personally,
I find the present result to only vaguely describe the abilities and
tasks required of my role as an energy modeler, and I think the
consensus of feedback so far indicates others do not find this terribly
representative either of their unique skill sets and abilities. A JTA
study developed to isolate and identify the different requirements and
duties for the 3 areas of energy modeling described above would be of
much more use for someone defining/ improving certifications, standards
and qualifications as well as defining the task at hand for solicitation
or hiring purposes.



PS: I notice in my final review of this that this commentary is coming
in a day late. I trust those interested will put this and any following
discussion from the lists in the right hands.

Nick-Caton's picture
Joined: 2011-09-30
Reputation: 805