Hi All
I do find ASHRAE Guideline 14 a little too hardcore for my basic understanding of statistics. I can plug any of the equations into Excel, but I?ve realised my statistics understanding is very limited! (I?m outed!)
We don?t actually have to work to G14 in the UK (probably good because my copy is a bit old). I finally realised I didn?t know enough after I?d been (lazily) using R2 in Excel on some monthly data. I thought that R2 > 0.9 was generally ok? yeah, it wasn?t.
So, are there any easy to understand resources available?
I?ve been messing around with the IMT as well. It?s been fun going back to DOS ?. This got me into daily methods, which leads to my next question. Is there any reason why there isn?t a daily calibration option specified in G14?
Many thanks!
Chris Yates
Dear Chris,
Kudos for appreciating a gap in your understanding. (I'm in your camp)
On the other hand, there are SO many variables in building operation that, short of a highly instrumented (and carefully calibrated) building for everything from lights to people to plug loads to HVAC - calibration is a fiction (and I'm confident that no such building exists). Daily calibration is a complete fiction, perhaps even a deception. On top of that, a "calibrated" model is just a moment in time; everything going forward is guaranteed to be different than during the calibration time period.
I think of "calibration" as more like a sensitivity analysis - determine which variables matter more and which matter less. GenOpt works nicely for that purpose https://github.com/lbl-srg/GenOpt
Jim Dirkes 1631 Acacia Drive NW Grand Rapids, MI 49504 - 616 450 8653
Coffee Conversation:
The "individual" is an impossible concept, conceived by the Enlightenment philosophers. It makes no sense to the Christian. In marriages, and families, in associations and friendships and religious orders, we are not individuals, but a communion of persons.
------- Original Message -------
Indirectly there probably isn?t a daily set of metrics in the Guideline since the simulation programs aren?t usually outputting daily results, but there?s no reason there couldn?t be one statistically.
You could make one if you had only daily utility data and had to aggregate the simulation results to daily totals, there isn?t a published target metric but you could still show that you calculated one and why you think it was a good or bad result.
DSE Mobile
I think I need to qualify this: informed by G14, but definitely not compliant with it! There is some allowance for repairing or ?healing? data, but when the data has a lot of holes or modes/ category variables then forget it. This is my case, but the client still wants some kind of representative simulation.
Monthly models can be garbage. School holidays cut across months at different times, combined heat and power is popular which complicates gas usage especially when metering is limited? did the heat by-product of electricity generation go to the building, or was it rejected? They can work for heating in our temperate climate, but not for cooling.
Nevertheless, we need some kind of representative simulation model. We can?t make any ECM qualifying claims but we can do something useful. This is where these methods can give you a lot of insight before you start modelling.
I hadn?t tried the IMT previously. We tend to have limited our regression analysis to monthly ?degree day? methods (your 2p model, I think). I plugged some project specific daily electricity data into the MVR example (multi variate regression) and it seemed to give decent CVRMSE (~2-3%) but low R2 (~0.7). However, it appeared to provide some insight on cooling usage (monthly 2p models are meaningless for this in the UK?s temperate climate).
I also made some 2p monthly models of gas. I thought these were good until I compared successive years. I guess this is were understanding a range of statistical indices is helpful.
Here?s the final rub, because the underlying data has so many inconsistencies that can only be made sense of with some regression models, it?s easier to ?calibrate? the simulation model to the regression models than the original data. But I may use 2p monthly for gas, daily MVR for electric?
So I need to ask if I have wondered completely off-piste with this!
Chris
I think Dru Crawley coined the phrase, "All models are wrong. Some are useful."
Chris, you highlighted an assortment of variables which are omnipresent, inconsistent and uncontrollable - so what, exactly, does your client expect? Is it a realistic expectation? For example, are they going to nail you to the wall when, inevitably, you are "wrong" next year?
I have not calibrated many models, but have been made more appreciative of all the uncontrollables by the ones I calibrated :(. All the statisticians know that there is always more than one solution which will result in a high R2 value or low CVRSME, so which is correct?
Rather than a calibrated model, lately I've been encouraging clients to consider one of the FDD platforms on top of their Building Automation System. Spending time and money to evaluate whether things are working properly makes more sense to me - it's "real life" vs a prediction. (Can't forget to mention thoughtful and thorough commissioning here; that's essential.)
ps, I love your thoughtful approach. You're setting a great example! One aspect of that is to reach out to the wider modeling community to gather input and feedback.
Jim Dirkes 1631 Acacia Drive NW Grand Rapids, MI 49504 - 616 450 8653
Coffee Conversation:
The "individual" is an impossible concept, conceived by the Enlightenment philosophers. It makes no sense to the Christian. In marriages, and families, in associations and friendships and religious orders, we are not individuals, but a communion of persons.
------- Original Message -------
You are allowed to make an M&V plan and then define what you did given the information available. In your case you are using G14 informatively, it wasn?t required that you follow it or report that you were using it.
The only thing I can add is maybe you can focus on validating some of the inputs ? if the utility data is spotty then document what you do know from the input side ? if you can show some of your most sensitive input variables are validated that would help give confidence that the outputs are also likely to be useful.
David
David S. Eldridge, Jr., P.E., BEMP, BEAP, HBDP, GGF/A
Direct: (847) 316-9224 | Mobile: (773) 490-5038
Grumman|Butkus Associates | 820 Davis Street, Suite 300 | Evanston, IL 60201
Energy Efficiency Consultants and Sustainable Design Engineers
grummanbutkus.com | Twitter | LinkedIn | Instagram
As a side note, Dru was probably quoting George Box:
https://en.wikipedia.org/wiki/All_models_are_wrong
Jason
George Box coined that one I think
I agree that better measurement is the key. It's surprising how completely
off the radar that is. There is an expectation of major interventions, new
facades, plant replacement, etc, but the basics get neglected.
I have a theory about using statistical models to assist with simulation
models. In theory, we should not just aim for low rmse, but coefficients
should be comparably close when performing the same regression on simulated
results.
Some more words of wisdom :)
https://en.wikipedia.org/wiki/There_are_unknown_unknowns?wprov=sfla1
On Tue, 14 Mar 2023, 19:35 David Eldridge,
wrote:
Thanks! Modelers will understand it readily. Clients need some explanation and level-setting.
Jim Dirkes 1631 Acacia Drive NW Grand Rapids, MI 49504 - 616 450 8653
Coffee Conversation:
The "individual" is an impossible concept, conceived by the Enlightenment philosophers. It makes no sense to the Christian. In marriages, and families, in associations and friendships and religious orders, we are not individuals, but a communion of persons.
------- Original Message -------
Hello,
This looks like a very interesting conversation. Unfortunately we once again get into a problem where not everyone is able to follow because the information is behind a paywall.
It's not crystal clear to me what is "hardcore" about it. It is about using notmalized mean bias (NMBE) + CV(RMSE) instead of the more familiar Rsquared?
If my limited memory serves, guideline 14 and the IPMVP protocol define, and explain, them properly.
The first critical piece is to use actual weather data and not try to calibrate to a tmy3 file.
I agree that daily (or hourly/subhourly) calibration is probably too optimistic, though again if memory serves the protocol has relaxed thresholds for that case.
My two cents is that you should first Downsample the utility data to monthly, unless you have less than 12 months of data without any major changes (ECMs or unusual activities).
If you have no period of 12 months without major external changes, massage the data. What's needed there depends on whether it's an ECM or unusual activity. If an ECM probably try to use 12 months before it If you can. If unusual, do a regression on the regular data, extrapolate the rest.
As it's been said before, the key point is to document whatever assumptions you've taken clearly.
Even when you have perfect data for 36 months of "regular operation" (whatever does that mean), and calibrate your model correctly,? you'll still be off in the next 12 months if nothing happened.
Never forget that you're dealing with 5000+ independent variables and trying to calibrate to 12month of one, or two, dependent variable(s) (utility data). It's a **grossly** overfit model. And that's why a careful audit is needed with existing buildings, so you can try to 1) get the history for the calibration period and 2) tune every variable you can so only the unknowns remain (typically nail the loads etc, leaving stuff that's not easily measured and very influential like infiltration when a blower door test is not practical)
Disclaimer: writing on my phone so zero external resources available and probably a lot of typos.
Best,
Julien
Get BlueMail for Android
I've found res and nonres to have 2 very different calibration paths. Just
my 2 cents on residential - I've found it helpful to calibrate to indoor
air temp in addition to monthly energy bills. Especially in low-income
households that have a high tolerance for thermal discomfort and price
anxiety. This combined with one or two granular time series of t-stat or
HVAC loads seem to be the only way to get a sense of what actually happens
in homes. The use case is transactive energy and community level smart
local energy markets providing distribution circuit ancillary services -
pretty specific thing - but will become increasingly important in many
national and subnational initiatives.
Marc
On Tue, Mar 14, 2023 at 2:39?PM Jim Dirkes via Bldg-sim <
--
CONFIDENTIALITY NOTE: The information in this email is intended only for
the use of the individual or entity named above. If you may have received
this message in error, please email or call the sender. If you are not the
intended recipient, any dissemination, distribution or copying is strictly
prohibited.
Hello Chris:
Thanks for weighing in on the IMT and G14.
First, in your original email you said:
As one of the co-developers of the IMT and a Committee member of G14-2023, I guess I'd better weigh-in. There's a lot to unpack in one email, but I'll try.
However, before I start I need to say that G14-2023 is due out this year after public review and comment. Hence, I'd suggest you get a copy and take a look to see if it is improved or not.
In addition, the discussion of the IMT and Guideline 14-2023 should be separated, since one was an ASHRAE Research Project, RP-1050, that had the goal of developing public regression code that others could take and use to put "ASHRAE under the hood" so to speak.
The reason for this was that there were many "methods" that were out there that were proprietary, or perhaps published, that did not give the use a computer code that they could use freely, and/or embed into their code to produce similar results -- in fact, that was the purpose and the wisdom of the PES that developed RP-1050, to produce public code that other could use.
I agree that your issues on hourly vs daily vs monthly is not addressed in Guideline 14, nor in the IMT. This was not on purpose, it just happened. The reason is pretty simple, over the last 20 years, things have evolved, and folks have hourly or sub-hourly data available, and there's no advice as to what's what, and or what to use.
This particular topic is actually being addressed by ASHRAE's new Guideline 45R (yes, another guideline) that seeks to address energy use, indoor environmental parameters (i.e., comfort, IAQ, daylighting, etc.) into one guideline. Stay tuned.
However, in the current G14-2023, there is little advice as to when to use hourly, daily or daily and what for. Only modeling guidelines for a generic time frame.
Finally, there are details about using the IMT (and ASHRAE RP-1093) to develop baseline and post-retrofit that are now contained in G14-2023. So, hopefully this will help.
I agree that the statistical parameters for G14-2023 are rather strict, and I would encourage you to contact the Committee members that developed these (I'll be happy to say who in a separate email). However, suffice it to say, these uncertainty stastics are complex.
As for the comment about modeling K-12 schools with monthly use, this has been addresed previously for monthly data in Margaret Fels 1986 Energy and Buildings article using a "PRISM plus UNDERBAR" method. Email me and I'll send you the articles.
Your reference to "GENOPT" makes he shiver, as I don't trust anything that I can't take apart and see how the parts work. So, I can't comment on this.
Yes, there are a lot of variables that influence whole-building energy use, however, that is the purpose of the analyst to tease these variables out of the analysis to see what's what.
"Healing" that data is needed since there are most likely "blips" in a multi-year data stream that are "out of bounds", and most likely should not be included in the analysis. Details of how this "period" was identified, and/or how the data were "healed' are necessary to explain to the authorities (in the case of an ESCO contract).
Monthly models can be garbage if there are big differences in the building operation from one month to the next. One example is university buildings, that have semester, non-semester and vacation periods that most likely need daily or even hourly data.
I'm nervous about "monthly degree-day" models since most commercial buildings don't behave as modeled by a 65F degree day model.
CVRMSE and adj R2 can go a long way in describing the accuracy of a model.
I agree that we need "some sort of representative simulation model"... however, none such exists at this time. So we need to rely on regression.
In regard to M&V plans, we've made many of these for ESCO projects in Texas that are mindful of resources and accuracyt, we'd be happy to share.
Hope this helps.
Jeff
Jeff S. Haberl, Ph.D., P.E.inactive, FASHRAE,FIBPSA We are like fluttering leaves on the branches of trees
Department of Architecture in the forests of the landscape that surrounds us.
Texas A&M University If we could, for just a moment, flutter together,
College Station, TX 77845-3581 We could lift the earth up to be a better place. JSH 2022
Office: 979-845-6507, Lab: 979-845-6065
Fax 979-862-2457
jhaberl at tamu.edu,www.esl.tamu.edu
PS.
I?m thankful that BLDG-SIM exists to allow all building analysis questions to be asked so as to maybe get an answer from the Grey- hairs as to what do.
We all stand together on open forums or we watch our industry demise.
Jeff
Jeff S. Haberl, Ph.D., P.E.inactive, FASHRAE, FIBPSA
Department of Architecture
Texas A&M University
College Station, Texas 77845-3581
Office: 979-845-6507, Lab: 979-845-6065
Fax: 970-862-2457, jhaberl at tamu.edu, www.esl.tamu.edu
Wow! This is the best bldg-sim discussion in years! Thanks to all who have
weighed in. My opinion is that calibration should be focused on the
decision that you are trying to make. What are you trying to estimate? Are
you trying to understand peak impacts? Or do you have a steep time of use
rate and care about value of energy savings? Or are you just trying to
reduce energy?
I?m a big fan of trying to calibrate to some kind of end use consumption
data and also looking at some hourly data to make sure you didn?t
completely blow the schedules. How many times have we figured out that
overnight consumption is wayyyy higher than expected?
Beyond that, it?s important to not get too precise, because you are likely
going too far. My favored expression on over fitting is from John von
Neumann, roughly: give me four parameters and I?ll draw you an elephant;
give me five and I?ll wiggle it?s trunk. I would beseech our junior
engineers, ?no wiggling the elephant?s trunk!?
Thanks to all who put the time in to give the rest of us guidance on how to
do this right!
On Tue, Mar 14, 2023 at 4:46 PM Haberl, Jeff via Bldg-sim <
Hello Justin.
Thanks for your email, in which you brought up the topic of peak demand. This has become VERY important in Texas, as you?ve read with our grid failure for ERCOT customers. As such, there are lots of folks simulating lots of prototypical household to get an answer state wide without asking how accurate is a BES regarding hourly peak demand? What?s the right weather file to use? Is it synchronous? Or asynchronous with other grid loads? Does my EQUEST or ENERGYPLUS model really represent demand? Dies the model REALLY represent the thermal mass?
All sorts of new issues for serious BES modelers.
Jeff
Jeff S. Haberl, Ph.D., P.E.inactive, FASHRAE, FIBPSA
Department of Architecture
Texas A&M University
College Station, Texas 77845-3581
Office: 979-845-6507, Lab: 979-845-6065
Fax: 970-862-2457, jhaberl at tamu.edu, www.esl.tamu.edu
Yep, not me. But I use it a lot.
On Tue, Mar 14, 2023, 4:19 PM Vikram Sami via Bldg-sim <
Hello all,
Just a couple thoughts from a random passerby. It?s rare to see this email
group this active :p
Now please bear with my 2 cents:
One thing that always bother me when working on energy model calibration in
the past was: why was cross validation (CV) not used?
One fundamental technique in Machine Learning to avoid overfitting is
through cross validation. Fitting a complex first-principle model is
similar to training a complex black-box model: high degree of freedom,
risks of overfitting. CV can be adopted to avoid incorrect local minima
during model training. There are multiple CV approaches for timeseries
model training.
And regarding predicting peak loads, etc. When training a model, the loss
function (equation that describes model accuracy) dictates how you want the
model to behave. Usually RMSE is used but you can customize it to make the
model behave in a certain way better. For example, something like a
compound loss function that combines prediction errors from both overall
predictions as well as peak prediction. You can adjust the weight for each
component to dictate which part is more important than another.
Thanks,
Shawn Shi
On Tue, Mar 14, 2023 at 7:41 PM Dru Crawley via Bldg-sim <
Thanks Shawn:
Good thoughts. Maybe outside my expertise, but I'll take a crack.
First, BES are constrained models in that they only allow the user to change cerain parameters and not others. Furthermore, the BES is a preconfigured model of a building that tries to include all the relevant heat transfer properties to the ambient conditions and the SYSTEM and PLANT reactions to these loads using user-defined inputs.
This situation only gets worse when on considers that even EP+ was influenced by the original LSPE architecture that was suggested for the Post Office program in the early design of the TACS program. Although EP+ has evolved way beyond the constraints of LSPE BES programs, the underlying code still has ghosts of this architecture in its algorithms. We recently published one of three new articles for STBE that updates the previous "Origins" articles we published. The first one is "Origins of whole-building energy simulations for high-performance commercial buildings: Contributions of NATEOUS, SHEP, TACS, CP-26, and RESPTK programs", by Jounghwan Ahn and myself. Two more articles to follow soon in STBE.
In the first article, thanks in part to Jason Glazier for dropping-off a pickup truck's worth of books when Robert Henniger retired, we outline how the early simulation programs can be traced to simulations of bomb shelters in the 1960s by Tamami Kusuda and Metin Lokmonhekim and the earliest FORTRAN programs that codified response factors by Tamami Kusuda (RESPTK). All of these books are now scanned and can be found on the onebuilding.org website.
In these books you see that what was done in the early days was done because of constraints in the computing hardware in the 1960s and into the 1970s. Unfortunately, ghosts of this kind of thinking still exist today, with a few exceptions, when we try to define a "thermal zone" as a node that maintains a known temperature given varying internal inputs and external R-C networks that connect the node to the exterior. So, although we've come a long way, we still have a long way to go to "cut all the stings" to the old LSPE constraints.
Second, there are prototypes of how to move forward into the future, but their "secrets" are highly guarded, which is not good for the general BES community. However, it is a fact of life and we have to live with it and put our minds to use to reverse engineer what the proprietary codes are doing so we can publish in the public domain so all can see what is done. EP+ is one effort that stands alone it what it has accomplished AND has accomplished in the public domain -- bravo.
However, there are spectacular proprietary codes that are doing breathtaking simulations that are hidden "behind the wall", and so it goes.
Examples include combined CFM and BES -- a daunting task. Another is full force raytracing and BES programs. Yet another is Urban Scale Building Energy Modeling (UBEM). All are paths into the future that are critical for BES modelers to learn and incorporate into the work. Finally, there are robo-simuilations that are now being attached to BIM, which is the future for BES. Yet, there is little discussion about public code, where to get it and whether or not it is worth a hoot.
So, I think BLDG-SIM has a role in providing a public space where "issues" such as the above can be discussed, and vetted in the public domain, so those who have the development money can get a second opinion about how they are spending state or federal money.
BLDG-SIM can also be seed-corn for student thesis topics since these are like hens teeth when thinking them up and then following through to publication
Just some thoughts.
Jeff
Jeff S. Haberl, Ph.D., P.E.inactive, FASHRAE,FIBPSA We are like fluttering leaves on the branches of trees
Department of Architecture in the forests of the landscape that surrounds us.
Texas A&M University If we could, for just a moment, flutter together,
College Station, TX 77845-3581 We could lift the earth up to be a better place. JSH 2022
Office: 979-845-6507, Lab: 979-845-6065
Fax 979-862-2457
jhaberl at tamu.edu,www.esl.tamu.edu
Hello Jeff,
Thank you kindly for the long reply and history lesson, it's interesting to
know BES originated from bomb shelters, there is one still operating not
far from where I live :)
As a gamer growing up, I've always wondered why BES is not voxel-based like
minecraft or why we are not using real-time ray-tracing from GPUs for solar
insolation computations.
Anyways, back to model training/calibration. Loss function (in ML training)
or cost function (in optimization) is the measuring stick telling the
optimization program how to assess the performance of the model. Usually
MSE/RMSE is used but it has its limitations, that's why in ML we sometimes
use log errors
to deal with outliers. Another example, in fraud detection, we customize
loss functions to penalize false negatives harsher than false positives.
Actually many years ago, someone mentioned custom cost function to deal
with peak loads for BES calibration, using ExcaliBEM:
https://unmethours.com/question/13182/genoptenergyplus-best-algorithm-for-peak-load-reduction-load-shift/
Now on cross-validation
(CV), it is a wrapper around each model training process to make sure the
model is not over-fitting. It is usually the first thing I check when
seeing people applying ML models.
In theory these two practices should not interfere with the bounded
parameter space.
Thanks,
Shawn
Dear Chris,
going back to your original question of "easy to understand resources" I
would say this guide by BPA (
https://www.bpa.gov/-/media/Aep/energy-efficiency/measurement-verification/3-bpa-mv-regression-reference-guide.pdf).
The document includes formulas and corresponding functions in spreadsheets.
In my experience, Variable-Base Degree-Days and Energy Signature models can
give essentially the same results if constructed with similar assumptions.
The simplest way (in my opinion) to handle piecewise linear shapes and/or
shifts in data is to include additional independent "dummy" variables that
can help subset the original dataset (e.g. weekday/weekend if you are using
daily data, or by day of the week) see this for a general explanation (
https://otexts.com/fpp2/useful-predictors.html).
In some cases you may need to include additional variables (e.g. solar
radiation for cooling, or enthalpy/HR if air handling processes are
involved, etc.) and then compare the models using a stepwise regression
approach, to check if the performance in terms of statistical indicators
RMSE, R2, CV(RMSE), etc. improves by adding additional independent
variables.
A tool that can automate some of the work (spreadsheet based, with a good
documentation and freely available) is ECAM (https://sbwconsulting.com/ecam/),
which enables modelling both Energy and Energy Signature. I tested it in
some case studies and it works quite well (at least with Excel 2016, not
sure about newer versions), even though the hourly model takes some time
(ECAM v7.0).
I can provide some indications on the use of the template in case or other
Python/R tools that can do a similar work.
Kind regards,
MM
Il giorno mer 15 mar 2023 alle ore 02:36 Shawn Shi via Bldg-sim <
bldg-sim at lists.onebuilding.org> ha scritto:
--
*Massimiliano Manfren, Eng. Ph.D.*
Webmail : massimiliano.manfren at gmail.com
University email : M.Manfren at soton.ac.uk
Skype: massimiliano.manfren
Hi Shawn,
Your ideas of bringing in gaming capabilities to improve building simulations are fascinating. And I completely agree on the need for cross-validation. Garbage-in-garbage-out sounds cute, but why do we even allow that garbage?
Have you considered applying for DOE funding to bring your concepts into E+? Or perhaps collaborate with the creator of Ladybug suite of tools? which I believe is also in the public domain?
Sincerely,
Smita
?
Smita Chandra Thomas
Founder and Principal
Energy Shrink, LLC
202-556-3369
Hi folks -- I have a few more comments. Building simulation models are good
for extrapolating in time (meaning to different weather conditions) and
space (when you make a change to a building design that cannot be
observed).
1. If you're just trying to extrapolate in time, a mathematical regression
approach or machine-learning approach may make more sense. My old team used
this approach to build out residential end use load shapes in Massachusetts
from metered data and we used a regression approach (with some terms that
reflected our building science knowledge) with cross-validation and a value
function, like what Shawn mentioned. We were focused on peak demand impacts
in this case, as well as overall energy consumption, so we weighted our
errors accordingly, to focus on peak demand. You can see the results of
this work here:
RES-1-Residential-Baseline-Study-Ph4-Comprehensive-Report-2020-04-02.pdf
(ma-eeac.org)
2. If you're trying to extrapolate in space, e.g. to a different building
design, then a building simulation model helps you out a lot, especially if
you start out with a measured stake in the ground for the things you care
most about, e.g. if you care about AC load during summer peaks or heating
load during winter peaks, then you need measurements of those to do a good
job. Frequently, we invest way more in complex data modeling and analysis,
when measurement and a simpler model or even a simplified calculation would
help much more. A basic model of (Measured data) * % savings from
engineering-based algorithm works well for single dimensional building
improvements, provided your savings estimation algorithm is decent.
On Wed, Mar 15, 2023 at 6:34?AM Smita Chandra Thomas via Bldg-sim <
I've had more responses back than I bargained for! Whilst I was hoping for
an easy answer, I think the takeaway is that it's not simple and it will
require some perseverance. Thank you to everybody.
Massimiliano, I do use ECAM and I highly recommend it. I'm surprised it
doesn't feature on this forum much. I use it a lot, but I don't use a lot
of it. I use it mainly for graphing (the box plot is excellent). I haven't
got the M&V tools working for me. The I-P unit-only restriction is a
problem for me. I don't know if this I-P unit restriction is why I cannot
run some of the models... negative degrees celsius might throw it. Here's a
3-minute movie for anybody who hasn't used it: https://youtu.be/3-KcEF5VbGo
I did take a look at the BPA book a while ago and perhaps it's time to
revisit it. I recall it was clearly laid out and wasn't too dense a read
(for a statistics book).
Shawn, CV is something I've never heard about. Now I've taken a primer on
youtube. It's beyond me right now, but I'm sure it will grow in use. I
always thought ML meant some kind of chat GPT thing but can see that it's
more about remixing model sources and test data and undertaking more
iterations using various options. We know iterations can make a big
difference.
I didn't realize this repository had been developed:
https://onebuilding.org/ Thanks again, Jeff.
Chris
On Wed, Mar 15, 2023 at 3:02?PM Justin Spencer via Bldg-sim <
Thanks Chris:
FYI, if anybody has problems finding the articles about RP150, RP1093, or other items I mentioned please let me know. I'll be happy to send them to you if you ask me.
BTW, in addition to the 4 STBE articles on "Origins", we also developed two papers on peak load calculations, which serve a useful purpose of reviewing the history of "loads" calculations, which for those of you born AC (after computers), were the other ? of the analysis of a building, namely the peak heating / cooling load to size the HVAC system, as well as the annual energy use prediction, which used to be a rough estimate:
Mao, C., Baltazar, J.C, Haberl, J. 2018. ?Comparison of Building Envelop Peak Load Design Methods?, Science and Technology for the Built Environment, Vol. 25, Issue 2 (October).
17.
Mao, C., Baltazar, J.C., Haberl, J.S., 2018. ?Literature Review of Building Peak Cooling Load Methods in the United States?, Science and Technology for the Built Environment, Volume 24, No. 3, pp. 228-237, ESL-PA-18-03-01 (March).
Otherwise, feel free to visit the ESL's website, where we have over 8,000 publications on-line, and lots of other stuff.
Energy Systems Laboratory (tamu.edu)
Jeff
Jeff S. Haberl, Ph.D., P.E.inactive, FASHRAE,FIBPSA We are like fluttering leaves on the branches of trees
Department of Architecture in the forests of the landscape that surrounds us.
Texas A&M University If we could, for just a moment, flutter together,
College Station, TX 77845-3581 We could lift the earth up to be a better place. JSH 2022
Office: 979-845-6507, Lab: 979-845-6065
Fax 979-862-2457
jhaberl at tamu.edu,www.esl.tamu.edu
BTW:
After reading this thread, again, I realized that I had responded mostly about simulation, and a bit about RP1050 and RP1093. Then I realized I left out a whole universe of papers that discussed the evaluation of the very best inverse models, namely the ASHRAE predictor competitions, old age I guess:
Miller, C., Balbach, C., Haberl, J. 2020. ?The ASHRAE Great Energy Predictor III Competition: Overview and Results?, Science and Technology for the Built Environment, Vol 26, No. 10, ESL-PA-20-07-02 (August).
Haberl, J., Thamilseran, S. 1998. ?Predicting Hourly Building Energy Use: The Great Energy Predictor Shootout II: Measuring Retrofit Savings,? ASHRAE Journal, Vol. 40, No. 1, pp. 49 - 56 ESL-TR-95-09-02(January).
Kreider, J. Haberl, J. 1994. ?Predicting Hourly Building Energy Usage: The Results of the 1993 Great Energy Predictor Shootout Identify the Most Accurate Method for Making Hourly Energy Use Predictions,? ASHRAE Journal, Vol. 35, No. 3, pp. 72 ? 81 ESL-PA-94-06-02 (June).
Now I know this is not fair talking about publications from 20+ years ago, but I'll be happy to send copies to those who request them.
In the first Predictor Competition in 1994, Jan Kreider (now passed away) and I proposed, developed and delivered a new idea that the internet could be used to have a competition. If you recall, this is before Google, so I was a bit of a issue. We had to use FTP sites, and emails to distribute the material for the competition.
The winner was amazing, Dr. David McKay from Cavendish labs in the UK. It is even more amazing because he was not even in the HVAC field, he was an astronomer. His submission smashed all the rest with its answer, because he REALLY knew what he was doing with his methods. In the competition, one year of hourly data for the Zachry building at TAMU was prepared so that every other week was included in the "testing" file, as well as another test regarding weather data. All contestants were asked to develop the inverse model and submit their results for the missing data. I collected the data and Jan ran the analysise. We then published the results in several ASHRAE publication, including a Journal paper and Transactions papers, as well as Transactions papers from each of the contestants. The reactions at the ASHRAE meeting were shocking. Lots of questions from members who were interested in learning something new, and lots of questions about members, one in particular, from members who would stand up and say "you mean you can predict the energy use of an institutional building with a regression model without visiting the building...utter nonsense...".
In the second competition right after the first, the same building was revisited and the rules changes slightly to see how well the models predicted results AFTER a retrofit was applied to the building. The winner was Robert Dodier and Gregor Henze at the University of Colorado. Results also included an ASHRAE Journal article and Transactions papers.
In the third competition, developed by Clayton Miller, Chris Balbach, myself and others in 2020, the idea was expanded to include 100s of data sets and a world-wide search for the best model. The winner was Matthew Motoki and Isamu Yamashita from Japan. Their model used an ensemble of models, including Light GBM and MLP models trained on different subsets of the data. Results also published in the ASHRAE STBE.
So, inconclusion, there's a bit out there in the ASHRAE publications going back 25+ years. However, there is lots to learn from those who are searching. Clever inverse models have their place, but only in the hands of master craftsmen (or craftpersons).
Jeff
Jeff S. Haberl, Ph.D., P.E.inactive, FASHRAE,FIBPSA We are like fluttering leaves on the branches of trees
Department of Architecture in the forests of the landscape that surrounds us.
Texas A&M University If we could, for just a moment, flutter together,
College Station, TX 77845-3581 We could lift the earth up to be a better place. JSH 2022
Office: 979-845-6507, Lab: 979-845-6065
Fax 979-862-2457
jhaberl at tamu.edu,www.esl.tamu.edu