ISO/IEC 21836:2020

DRAFT INTERNATIONAL STANDARD
ISO/IEC DIS 21836
ISO/IEC JTC 1/SC 39 Secretariat: ANSI
Voting begins on: Voting terminates on:
2019-03-07 2019-05-30
Information Technology — Data Centres — Server Energy
Effectiveness Metric
Technologies de l'information — Centres de données — Grandeurs de mesure de l'efficacité énergétique des
serveurs
ICS: 35.020
THIS DOCUMENT IS A DRAFT CIRCULATED
FOR COMMENT AND APPROVAL. IT IS
THEREFORE SUBJECT TO CHANGE AND MAY
NOT BE REFERRED TO AS AN INTERNATIONAL
STANDARD UNTIL PUBLISHED AS SUCH.
IN ADDITION TO THEIR EVALUATION AS
BEING ACCEPTABLE FOR INDUSTRIAL,
This document is circulated as received from the committee secretariat.
TECHNOLOGICAL, COMMERCIAL AND
USER PURPOSES, DRAFT INTERNATIONAL
STANDARDS MAY ON OCCASION HAVE TO
BE CONSIDERED IN THE LIGHT OF THEIR
POTENTIAL TO BECOME STANDARDS TO
WHICH REFERENCE MAY BE MADE IN
Reference number
NATIONAL REGULATIONS.
ISO/IEC DIS 21836:2019(E)
RECIPIENTS OF THIS DRAFT ARE INVITED
TO SUBMIT, WITH THEIR COMMENTS,
NOTIFICATION OF ANY RELEVANT PATENT
RIGHTS OF WHICH THEY ARE AWARE AND TO
©
PROVIDE SUPPORTING DOCUMENTATION. ISO/IEC 2019

---------------------- Page: 1 ----------------------
ISO/IEC DIS 21836:2019(E)

COPYRIGHT PROTECTED DOCUMENT
© ISO/IEC 2019
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may
be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting
on the internet or an intranet, without prior written permission. Permission can be requested from either ISO at the address
below or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: +41 22 749 01 11
Fax: +41 22 749 09 47
Email: copyright@iso.org
Website: www.iso.org
Published in Switzerland
ii © ISO/IEC 2019 – All rights reserved

---------------------- Page: 2 ----------------------
ISO/IEC DIS 21836:2019(E)
40 CONTENTS
41
42 Foreword . 6
43 Introduction . 7
44 1 Scope . 8
45 2 Normative references . 8
46 3 Definitions and abbreviations . 8
47 3.1 Definitions . 8
48 3.2 Abbreviations . 17
49 4 Applicability of Server Energy Effectiveness Metric (SEEM) . 19
50 4.1 General . 19
51 4.2 Applicability of SERTv2 . 19
52 4.3 Determination of applicability for an “implementer-specified” metric . 19
53 5 Determination of Server Energy Effectiveness Metric (SEEM) . 20
54 5.1 General . 20
55 5.2 Power supply requirements . 20
56 5.3 SERTv2 active energy effectiveness metric . 20
57 5.4 Determination of “implementer-specified” metrics . 22
58 General . 22
59 “Implementer-specified” active metrics . 22
60 “Implementer-specified” idle metric . 23
61 6 SEEM implementation . 24
62 6.1 General . 24
63 6.2 Implementation steps for SEEM . 24
64 6.3 Implementation steps for SERTv2 . 24
65 6.4 Implementation steps for “implementer-specified” metrics . 25
66 6.5 Threshold selection . 25
67 7 Server Testing . 26
68 7.1 Configuration . 26
69 7.2 Environment . 26
70 7.3 Power analyser requirements . 26
71 7.4 Power requirements . 26
72 7.5 SEEM idle power test method . 26
73 7.6 Technical support . 27
74 Technical support requirements . 27
75 SPEC SERTv2 technical support . 27
76 7.7 Server product family and special configuration testing . 28
77 General . 28
78 Single configuration testing . 29
79 SERTv2 product family and special configurations . 29
80 Product family and special configurations for “implementer-specified”
81 metrics . 31
82 7.8 “Implementer-specified” metric creation testing. 31
83 General . 31
84 Functionality and reproducibility testing . 31

iii
© ISO/IEC 2019 – All rights reserved

---------------------- Page: 3 ----------------------
ISO/IEC DIS 21836:2019(E)
85 Real-world energy savings testing . 32
86 8 Reporting SEEM . 33
87 8.1 Server categories . 33
88 General . 33
89 SERTv2 server category definitions . 33
90 “Implementer-specified” metric server categories . 36
91 8.2 Documentation . 36
92 General . 36
93 Implementation documentation . 36
94 End user documentation . 37
95 9 Minor update acceptance testing . 40
96 9.1 General . 40
97 9.2 SEEM minor update testing and requirements . 40
98 SEEM component versions . 40
99 Power analysers . 40
100 Temperature sensors . 42
101 9.3 SERTv2 minor update testing and requirements . 42
102 Component versions and allowed updates . 42
103 Requirements for new software and hardware compliance . 42
104 9.4 “Implementer-specified” metric minor update requirements . 51
105 (normative) SERTv2 pre-approved CPU architectures, operating systems, and
106 Java Virtual Machines. 53
107 (normative) SEEM pre-approved power analysers and temperature sensors . 64
108 (normative) SERTv2 required documentation fields . 66
109 (normative) Deployed power assessment . 68
110 D.1 General . 68
111 D.2 Deployed power calculations . 68
112 D.3 Deployed power visualization . 69
113 D.3.1 Charting deployed power . 69
114 D.3.2 Creating a deployed power ranking table . 71
115 D.4 Deployed power analysis . 73
116 D.4.1 General . 73
117 D.4.2 Interpreting deployed power chart results . 73
118 D.4.3 Interpreting deployed power ranking tables . 74
119 Bibliography . 76
120
121 List of tables
122
123 Table 1 - SERTv2 server categories by form factor and function (Option 1) . 33
124 Table 2 - SERTv2 server categories by CPU architecture class (Option 2) . 34
125 Table A.1 - Pre-approved CPU Architectures, Operating Systems, and Java Virtual
126 Machines . 53
127 Table B.1 – Pre-approved power analysers . 64
128 Table B.2 – Pre-approved temperature sensors . 65
129 Table D.1 - Example Ranking Analysis Table . 73

iv © ISO/IEC 2019 – All rights reserved

---------------------- Page: 4 ----------------------
ISO/IEC DIS 21836:2019(E)
130
131 List of figures
132
133 Figure 1 - Example SERTv2 HTML Result Output . 39
134 Figure 2 - Example SERTv2 Text Result Output . 39
135 Figure D.1 - Example Ideal Deployed Power Chart . 71
136 Figure D.2 - Example Deployed Idle Power Chart. 71
137

v
© ISO/IEC 2019 – All rights reserved

---------------------- Page: 5 ----------------------
ISO/IEC DIS 21836:2019(E)
138 Foreword
139 ISO (the International Organization for Standardization) is a worldwide federation of
140 national standards bodies (ISO member bodies). The work of preparing International
141 Standards is normally carried out through ISO technical committees. Each member body
142 interested in a subject for which a technical committee has been established has the right to
143 be represented on that committee. International organizations, governmental and non-
144 governmental, in liaison with ISO, also take part in the work. ISO collaborates closely with
145 the International Electrotechnical Commission (IEC) on all matters of electrotechnical
146 standardization.
147 The procedures used to develop this document and those intended for its further
148 maintenance are described in the ISO/IEC Directives, Part 1. In particular the different
149 approval criteria needed for the different types of ISO documents should be noted. This
150 document was drafted in accordance with the editorial rules of the ISO/IEC Directives, Part
151 2 (see www.iso.org/directives).
152 Attention is drawn to the possibility that some of the elements of this document may be the
153 subject of patent rights. ISO shall not be held responsible for identifying any or all such
154 patent rights. Details of any patent rights identified during the development of the
155 document will be in the Introduction and/or on the ISO list of patent declarations received
156 (see https://www.iso.org/iso-standards-and-patents.html).
157 Any trade name used in this document is information given for the convenience of users and
158 does not constitute an endorsement.
159 For an explanation on the meaning of ISO specific terms and expressions related to
160 conformity assessment, as well as information about ISO’s adherence to the WTO principles
161 in the Technical Barriers to Trade (TBT) see, https://www.iso.org/foreword-
162 supplementary-information.html.
163 The committee responsible for this document is ISO/IEC JTC 1, Information technology, SC
164 39, Sustainability for and by Information Technology.
165

vi © ISO/IEC 2019 – All rights reserved

---------------------- Page: 6 ----------------------
ISO/IEC DIS 21836:2019(E)
166 Introduction
167 The global economy is now totally reliant on information and communication technologies
168 (ICT) and the associated generation, transmission, dissemination, computation, and storage
169 of digital data. While the Internet backbone carries the traffic, it is data centres which find
170 themselves at the nodes and hubs of a wide variety of both private enterprise and
171 shared/collocation facilities. With the large and continually increasing data capacity
172 demands placed on data centres worldwide, efficient use of data centre energy is an
173 extremely important strategy to manage environmental, cost, electrical grid capacity and
174 other impacts.
175 The ISO/IEC 30134 series of International Standards specify data centre energy
176 effectiveness KPIs to help data centre operators measure and improve specific aspects of
177 data centre energy effectiveness. ISO/IEC 30134-4, in particular, defines a method to
178 measure the peak capacity and utilization of servers operating in a data centre using
179 operator selected benchmarks. It, however, does not provide a method of comparing
180 individual server energy effectiveness across data centres and as stated in ISO/IEC 30134-
181 4, “should not be used to set regulation for a data centre or individual server”. There is
182 stakeholder demand for an international standard to measure the energy effectiveness of
183 servers before procurement and installation, particularly for use in worldwide server
184 energy effectiveness regulations and programs.
185 This standard provides a server energy effectiveness metric (SEEM) to measure and report
186 the energy effectiveness of specific server designs and configurations. This document will
187 be useful to stakeholders, including vendors, users and governments, from the design
188 verification testing phase all the way through compliance verification, procurement and
189 operation. Organizations that wish to establish compliance or reporting programs will find
190 that the test methods and scoring specified in ISO/IEC 21836 will save them significant time
191 and effort in implementing such programs. Standardization across such programs will allow
192 vendors to comply to stakeholder requirements more quickly and efficiently.
193 For applicable servers, ISO/IEC 21836 builds upon the widely-adopted Server Efficiency
194 Rating Tool™ (SERT), developed by the Standard Performance Evaluation Corporation
®
195 (SPEC ), as the energy effectiveness metric and test method. For servers where SERT is not
196 applicable, this standard provides requirements for the creation of alternate server energy
197 effectiveness metrics, referred to as “implementer-specified” metrics.
198
199

vii
© ISO/IEC 2019 – All rights reserved

---------------------- Page: 7 ----------------------
DRAFT INTERNATIONAL STANDARD ISO/IEC DIS 21836:2019(E)
200 Information technology – Data centres – Server Energy
201 Effectiveness Metric
202
203 1 Scope
204 This document specifies a measurement method to assess and report the energy effectiveness of
205 a computer server. This standard does not set any pass/fail criteria for servers.
206 2 Normative references
207 The following documents are referred to in the text in such a way that some or all of their content
208 constitutes requirements of this document. For dated references, only the edition cited applies.
209 For undated references, the latest edition of the referenced document (including any
210 amendments) applies.
211 SPEC Server Efficiency Rating Tool (SERT) version 2.x.x
212 SPEC PTDaemon™ version 1.8.1 and later
213 SPEC SERT Run and Reporting Rules version 2.0.1 (20170929)
214 SPEC Client Configuration XML (https://www.spec.org/sert2/SERT-JVM_Options-2.0.html)
215 3 Definitions and abbreviations
216 3.1 Definitions
217 For the purposes of this document, the following terms and definitions apply.
218 ISO and IEC maintain terminological databases for use in standardization at the following
219 addresses:
220 • IEC Electropedia: http://www.electropedia.org/
221 • ISO Online browsing platform: http://www.iso.org/obp
222
223 64-bit server
224 server which includes a CPU which has data path widths, memory addressing, registers, and other
225 architectural features which are 64-bits wide
226
227 active state
228 operational state in which the server is carrying out data processing in response to external
229 requests
230 Note 1 to entry: Example: active processing, data seeking/retrieval from memory, cache, internal/external
231 storage while awaiting further input over the network

© ISO/IEC 2019 – All rights reserved 1

---------------------- Page: 8 ----------------------
ISO/IEC DIS 21836:2019(E)
232
233 active state power
234 average server power, in watts, when server is performing data processing
235
236 auxiliary processing accelerator
237 additional compute device installed in the computer server that handles parallelized workloads
238 in conjunction with the CPU
239
240 blade chassis
241 enclosure that contains shared resources for the operation of blade servers, blade storage, and
242 other blade form-factor devices
243 Note 1 to entry: Shared resources provided by a chassis may include power supplies, data storage and
244 hardware for DC power distribution, thermal management, system management, and network services
245
246 blade server
247 server that is designed for use in a blade chassis
248 Note 1 to entry: A blade server is a high-density device that functions as an independent server and includes
249 at least one processor and system memory, but is dependent upon shared blade chassis resources (e.g.
250 power supplies, cooling) for operation
251
252 blade storage
253 storage device that is designed for use in a blade chassis that is dependent upon shared blade
254 chassis resources, like power supplies or cooling, for operation.
255
256 buffered memory
257 circuitry between the server’s memory and memory controller to either increase memory
258 capacity, increase bandwidth, and/or reduce the electrical load on the memory controller
259
260 coefficient of determination
261 statistic used to determine the strength of a fit between a mathematical model and a set of
262 observed data values
263 [SOURCE: ISO 15551-1:2015, 3.26, Note 1 has been removed]
264
265 coefficient of variation
266 standard deviation divided by the mean
267 [SOURCE: ISO 3534-1:2006]

© ISO/IEC 2019 – All rights reserved
2

---------------------- Page: 9 ----------------------
ISO/IEC DIS 21836:2019(E)
268
269 configuration
270 interrelated functional and physical characteristics of a product specified in product configuration
271 information
272 Note 1 to entry: This standard employs the following configurations: low-end, high-end and typical
273 Note 2 to entry: For server products, a configuration is one of many possible combinations of components
274 including CPU, storage devices, memory size and capacity and input/output devices for a single server
275 product within a larger product family. There are a large number of possible configurations within a
276 product family.
277 [SOURCE: ISO 17599:2015, 3.15 – modified, Note 1 and Note 2 have been added]
278
279 core
280 component of a processor which can independently receive instructions and takes actions or
281 performs calculations in response
282
283 CPU
284 a central processing element with functions for interpreting and executing instructions.
285 Note 1 to entry: In this standard, cache memory is included with the CPU
286 Note 2 to entry: This standard uses the terms CPU and processor interchangeably
287 [SOURCE: ISO 14576:1999, 2.1.9 – modified, Definition is shortened, Note 1 and Note 2 have been added]
288
289 CPU architecture
290 CPU design with significant similarities to other CPU architectures within the same CPU
291 architecture class
292 Note 1 to entry: CPU architectures are used to create CPU models which are often released in a similar
293 timeframe
®
294 Note 2 to entry: Examples of CPU architectures in the same CPU architecture class are Intel Haswell, Intel
®
295 Broadwell, and Intel Skylake, or separately, AMD Bulldozer, AMD Piledriver, and AMD Steamroller
296
297 CPU architecture class
298 group of one or more CPU architectures which share the same instruction set architecture and in
299 which newer architectures designs are derived from previous architecture designs
300 Note 1 to entry: Within a CPU architecture class, the initial CPU architecture is, for the most part, a new
301 design, and subsequent CPU architectures are derived from preceding CPU architectures
® ®
302 Note 2 to entry: Examples of different CPU architecture classes are ARMv8-A and AMD EPYC
303 Note 3 to entry: In certain cases, software programs need to be recompiled for use with different CPU
304 architecture classes
305
306 CPU model
307 specific CPU that is sold in the marketplace

© ISO/IEC 2019 – All rights reserved 3

---------------------- Page: 10 ----------------------
ISO/IEC DIS 21836:2019(E)
308 Note 1 to entry: All CPUs of the same model share the same technical characteristics, such as core
309 frequencies and core counts, and can be used interchangeably
310 Note 2 to entry: Examples of different CPU models are AMD EYPC 7601, AMD EYPC 7251 and Intel Xeon
311 Platinum 8180
312
313 CPU nominal frequency
314 the CPU core clock frequency which is the main frequency used in naming, marketing, and selling
315 the CPU
316
317 data averaging internal
318 for a power analyser, the time period over which all samples captured by the high-speed sampling
319 electronics of the analyser are averaged to provide a set of measured data
320
321 double data rate
322 transfers data on the rising and falling edges of the memory clock signal, resulting in twice the
323 memory bandwidth capacity at a specific clock frequency
324
325 end user
326 person or persons who will ultimately be using the system for its intended purpose
327 Note 1 to entry: for the purposes of this standard, the end user is the entity applying for certification of a
328 server model to a SEEM compliant regulation or program. For example, if server manufacturer A was
329 submitting a server model to ENERGY STAR for certification, server manufacturer A would be the end user.
330 [SOURCE: ISO/IEC/IEEE 24765:2010, 3.990, Note 1 has been added]
331
332 energy effectiveness
333 measure of the amount of data processing performed for a given amount of energy consumed
334 Note 1 to entry: for the purposes of this standard energy effectiveness is equivalent to the term energy
335 efficiency as used in server compliance regulations and programs such as ENERGY STAR®.
336
337 expansion auxiliary processing accelerator
338 auxiliary processing accelerator that is an add-in card installed in a general-purpose add-in
339 expansion slot.
340 Note 1 to entry: An expansion APA add-in card may include one or more APAs and/or separate, dedicated
341 removable switches
342 Note 2 to entry: example is a GPGPU installed in a PCI-e slot
343
344 fair use
345 uses of content that are considered valid defences to copyright infringement, such as for criticism
346 or educational purposes

© ISO/IEC 2019 – All rights reserved
4

---------------------- Page: 11 ----------------------
ISO/IEC DIS 21836:2019(E)
347 Note 1 to entry: For the purposes of this standard, fair use refers to the guidelines the developer of a metric
348 has specified related to how results can be used
349 [SOURCE: ISO 19153:2014, 4.14 – modified, original Note 1 remove, Note 1 added]
350
351 fully fault tolerant server
352 computer server that is designed with complete hardware redundancy, in which every computing
353 component is replicated between two nodes running identical and concurrent workloads
354 Note 1 to entry: A fully fault tolerant server uses two systems to simultaneously and repetitively run a
355 single workload for continuous availability in a mission critical application
356 Note 2 to entry: An example of a fault tolerant server; if one node fails or needs repair, the second node
357 can run the workload alone to avoid downtime
358
359 hardware threads
360 in a CPU core, the number of fully independent instruction streams which can be executed through
361 SMT
362
363 high performance computing (HPC) system
364 computing system which is designed, marketed, sold, and optimized to execute highly parallel
365 applications for high performance, deep learning, or artificial intelligence applications
366 Note 1 to entry: HPC systems consist of multiple clustered servers, primarily for increased computational
367 capability, high speed inter-processing interconnects, large and high bandwidth memory capability and
368 often accelerators such as GPGPUs or FPGAs
369 Note 2 to entry: HPC systems may be purposely built or assembled from more commonly available computer
370 servers
371
372 high-end configuration
373 server equipped with a specific selection of high performance components, which is required to
374 be tested as part of measuring a server product family
375
376 idle state
377 operational mode in which the OS and other software have completed loading, the server is
378 capable of completing workload transactions, but no active workload transactions are requested
379 or pending by the system
380 Note 1 to entry: In the idle state, the server is operational, but not performing any useful data
381 processing
382 Note 2 to entry: For systems where Advanced Configuration and Power Interface (ACPI) has
383 been implemented, idle state is the ACPI G0 global state and S0 sleep state.
384
385 idle state power
386 average server power, in watts, when in idle state

© ISO/IEC 2019 – All rights reserved 5

---------------------- Page: 12 ----------------------
ISO/IEC DIS 21836:2019(E)
387 Note 1 to entry: SERT provides a standard way to measure the idle power of a server, which is included
388 with the result output, and is in addition to power measurement while the server is actively performing data
389 processing
390
391 implementer
392 entity that transforms specified designs into their physical realization
393 Note 1 to entry: For the purposes of this standard, implementer is the entity which creates a selection or
394 procurement program based on SEEM.
395 [SOURCE: IEC 62279:2015, 3.1.15, Note 1 added]
396
397 integrated auxiliary processing accelerator
398 auxiliary proces
...