SIST EN 303 470 V1.1.1:2019

2003-01.Slovenski inštitut za standardizacijo. Razmnoževanje celote ali delov tega standarda ni dovoljeno.Environmental Engineering (EE) - Energy Efficiency measurement methodology and metrics for servers35.020Informacijska tehnika in tehnologija na splošnoInformation technology (IT) in general27.015VSORãQREnergy efficiency. Energy conservation in generalICS:Ta slovenski standard je istoveten z:ETSI EN 303 470 V1.1.1 (2019-03)SIST EN 303 470 V1.1.1:2019en01-maj-2019SIST EN 303 470 V1.1.1:2019SLOVENSKI

ETSI EN 303 470 V1.1.1 (2019-03) Environmental Engineering (EE); Energy Efficiency measurement methodology and metrics for servers

Reference DEN/EE-EEPS24 Keywords energy efficiency, ICT, server, sustainability ETSI 650 Route des Lucioles F-06921 Sophia Antipolis Cedex - FRANCE

Siret N° 348 623 562 00017 - NAF 742 C Association à but non lucratif enregistrée à la Sous-Préfecture de Grasse (06) N° 7803/88

Important notice The present document can be downloaded from: http://www.etsi.org/standards-search The present document may be made available in electronic versions and/or in print. The content of any electronic and/or print versions of the present document shall not be modified without the prior written authorization of ETSI. In case of any existing or perceived difference in contents between such versions and/or in print, the prevailing version of an ETSI deliverable is the one made publicly available in PDF format at www.etsi.org/deliver. Users of the present document should be aware that the document may be subject to revision or change of status. Information on the current status of this and other ETSI documents is available at https://portal.etsi.org/TB/ETSIDeliverableStatus.aspx If you find errors in the present document, please send your comment to one of the following services: https://portal.etsi.org/People/CommiteeSupportStaff.aspx Copyright Notification No part may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying and microfilm except as authorized by written permission of ETSI. The content of the PDF version shall not be modified without the written authorization of ETSI. The copyright and the foregoing restriction extend to reproduction in all media.

DECTTM, PLUGTESTSTM, UMTSTM and the ETSI logo are trademarks of ETSI registered for the benefit of its Members. 3GPPTM and LTETM are trademarks of ETSI registered for the benefit of its Members and of the 3GPP Organizational Partners. oneM2M™ logo is a trademark of ETSI registered for the benefit of its Members and of the oneM2M Partners. GSM® and the GSM logo are trademarks registered and owned by the GSM Association. SIST EN 303 470 V1.1.1:2019

ETSI ETSI EN 303 470 V1.1.1 (2019-03) 3 Contents Intellectual Property Rights ................................................................................................................................ 6 Foreword ............................................................................................................................................................. 6 Modal verbs terminology .................................................................................................................................... 6 Introduction ........................................................................................................................................................ 6 1 Scope ........................................................................................................................................................ 8 2 References ................................................................................................................................................ 9 2.1 Normative references ......................................................................................................................................... 9 2.2 Informative references ........................................................................................................................................ 9 3 Definition of terms, symbols and abbreviations ..................................................................................... 10 3.1 Terms ................................................................................................................................................................ 10 3.2 Symbols ............................................................................................................................................................ 14 3.3 Abbreviations ................................................................................................................................................... 14 4 Server product categories and representative product family configurations ........................................ 15 4.1 General ............................................................................................................................................................. 15 4.2 Applications and metric applicability ............................................................................................................... 15 4.3 Computer servers .............................................................................................................................................. 16 4.3.1 General requirements .................................................................................................................................. 16 4.3.2 Form-factors ............................................................................................................................................... 16 4.4 Computer server product categories ................................................................................................................. 16 4.5 Server product family configuration................................................................................................................. 17 4.5.1 General ........................................................................................................................................................ 17 4.5.2 "High-end" performance configuration ...................................................................................................... 17 4.5.3 "Low-end" performance configuration ....................................................................................................... 17 5 Metrics .................................................................................................................................................... 18 5.1 Active state metric ............................................................................................................................................ 18 5.1.1 Worklets ...................................................................................................................................................... 18 5.1.2 Formulae ..................................................................................................................................................... 18 5.1.2.1 General .................................................................................................................................................. 18 5.1.2.2 Active State Metric definition ............................................................................................................... 18 5.1.3 Weightings .................................................................................................................................................. 19 5.2 Idle state metric ................................................................................................................................................ 19 6 Test setup ................................................................................................................................................ 20 6.1 General ............................................................................................................................................................. 20 6.2 Input power ...................................................................................................................................................... 21 6.3 Environmental conditions ................................................................................................................................. 21 6.3.1 Ambient temperature .................................................................................................................................. 21 6.3.2 Relative humidity ........................................................................................................................................ 21 6.4 Power analyser ................................................................................................................................................. 21 6.5 Temperature sensor .......................................................................................................................................... 22 6.6 Active state test tool ......................................................................................................................................... 22 6.7 Controller system ............................................................................................................................................. 22 6.8 General SERT™ requirements ......................................................................................................................... 22 7 Equipment Under Test (EUT) ................................................................................................................ 23 7.1 Configuration ................................................................................................................................................... 23 7.2 Test procedure .................................................................................................................................................. 24 8 Measurement .......................................................................................................................................... 25 8.1 Measurement for active state ............................................................................................................................ 25 8.2 Sensitivity analysis ........................................................................................................................................... 25 8.3 Measurement for power supply ........................................................................................................................ 26 8.3.1 Measurement for internal power supply ..................................................................................................... 26 8.3.2 Measurement for test board power supply .................................................................................................. 26 SIST EN 303 470 V1.1.1:2019

ETSI ETSI EN 303 470 V1.1.1 (2019-03) 4 8.3.2.1 General .................................................................................................................................................. 26 8.3.2.2 Test loads .............................................................................................................................................. 26 8.3.2.3 Test leads and wiring ............................................................................................................................ 26 8.3.2.4 Warm up time ........................................................................................................................................ 26 8.3.2.5 Power measurements ............................................................................................................................. 27 9 Measurement report ................................................................................................................................ 27 Annex A (normative): Resilient server requirements ....................................................................... 28 A.1 Reliability, Availability and Serviceability (RAS) features ................................................................... 28 A.2 Reliability, Availability and Serviceability (RAS) requirements ........................................................... 28 Annex B (informative): Deployed Power Assessment ......................................................................... 30 B.1 Overview ................................................................................................................................................ 30 B.2 Determining the number of deployed servers ........................................................................................ 30 B.2.1 General ............................................................................................................................................................. 30 B.2.2 Establishing target performance ....................................................................................................................... 31 B.2.3 Weighting factors ............................................................................................................................................. 31 Annex C (informative): Alternative calculation .................................................................................. 33 Annex D (informative): Bibliography ................................................................................................... 34 History .............................................................................................................................................................. 35

ETSI ETSI EN 303 470 V1.1.1 (2019-03) 5 List of tables Table 1: Configuration of EUT .........................................................................................................................................23 Table 2: Test configuration of EUT ..................................................................................................................................24 Table 3: Measurement of active state ................................................................................................................................25 Table 4: Measurement of internal power supply ...............................................................................................................26 Table A.1: Resilient Server requirements..........................................................................................................................29

ETSI ETSI EN 303 470 V1.1.1 (2019-03) 6 Intellectual Property Rights Essential patents

IPRs essential or potentially essential to normative deliverables may have been declared to ETSI. The information pertaining to these essential IPRs, if any, is publicly available for ETSI members and non-members, and can be found in ETSI SR 000 314: "Intellectual Property Rights (IPRs); Essential, or potentially Essential, IPRs notified to ETSI in respect of ETSI standards", which is available from the ETSI Secretariat. Latest updates are available on the ETSI Web server (https://ipr.etsi.org/). Pursuant to the ETSI IPR Policy, no investigation, including IPR searches, has been carried out by ETSI. No guarantee can be given as to the existence of other IPRs not referenced in ETSI SR 000 314 (or the updates on the ETSI Web server) which are, or may be, or may become, essential to the present document. Trademarks The present document may include trademarks and/or tradenames which are asserted and/or registered by their owners. ETSI claims no ownership of these except for any which are indicated as being the property of ETSI, and conveys no right to use or reproduce any trademark and/or tradename. Mention of those trademarks in the present document does not constitute an endorsement by ETSI of products, services or organizations associated with those trademarks. Foreword This European Standard (EN) has been produced by ETSI Technical Committee Environmental Engineering (EE).

National transposition dates Date of adoption of this EN: 11 March 2019 Date of latest announcement of this EN (doa): 30 June 2019 Date of latest publication of new National Standard or endorsement of this EN (dop/e):

31 December 2019 Date of withdrawal of any conflicting National Standard (dow): 31 December 2019

Modal verbs terminology In the present document "shall", "shall not", "should", "should not", "may", "need not", "will", "will not", "can" and "cannot" are to be interpreted as described in clause 3.2 of the ETSI Drafting Rules (Verbal forms for the expression of provisions). "must" and "must not" are NOT allowed in ETSI deliverables except when used in direct citation. Introduction The present document specifies a metric for the assessment of energy efficiency of computer servers using reliable, accurate and reproducible measurement methods, which take into account the recognized state of the art. The present document formalizes the tools, conditions and calculations used to generate a single figure of merit of a single computer server representing its relative efficiency and power impact. The metric is targeted for use as a tool in the selection process of servers to be provisioned. SIST EN 303 470 V1.1.1:2019

ETSI ETSI EN 303 470 V1.1.1 (2019-03) 7 For comparisons, evaluations should be conducted across similar server types or categories. The efficiency metric is targeted for use in a pass/fail selection process by differentiating the ability of servers to be provisioned for general purpose operations. The present document does not prescribe the levels or values for acceptance but prescribes a standard method of evaluation that energy efficiency programs would use to establish such criteria. As there are many operational deployments of servers resulting in a range of specialized equipment and configurations for a single server product, a metric that evaluates provisioning impacts to general purpose operations may not be applicable. ICT equipment and servers in particular, are generally customized and commissioned on site for deployment. As with most IT equipment, new technologies are regularly introduced, which may require product level customization or an industry wide tool upgrade to more appropriately represent the efficiency of the servers. The present document categorizes servers to address applicability, configuration groupings to represent a family of servers to address the broad range of custom configurations possible within each server product family, and tool revision control to ensure comparability and consistency of the resulting metric value.

The present document is based upon the Server Efficiency Rating Tool™ (SERT™) of the Standard Performance Evaluation Corporation (SPEC) and takes into account: • the Eco-design Technical Assistance Study on Standards for ErP Lot 9 Enterprise Servers and Enterprise Data Storage; • activity related to the analysis of output of Server Efficiency Rating Tool (SERT™) measurements and deployed power by The Green Grid; • ENERGY STAR® for Computer Servers [i.2]. The present document defines energy efficiency metrics and measurement methodology for server equipment under standardization mandate M/462 of the European Commission [i.3].

ETSI ETSI EN 303 470 V1.1.1 (2019-03) 8 1 Scope The present document specifies a metric using the Server Efficiency Rating Tool (SERT™), test conditions and product family configuration for the assessment of energy efficiency of computer servers using reliable, accurate and reproducible measurement methods. The metric applies to general purpose computer servers with up to four processor sockets and with their own dedicated power supply. NOTE 1: The term "socket" also applies to design in which processors are installed without sockets (e.g. soldered products).

The metric applies to a computer server model and to a computer server product family, including type and count of CPU, memory, storage, power supplies, cooling (e.g. fans) and any other add-on hardware expected to be present when deployed.

The present document defines: • an energy efficiency metric to support procurement or market entry requirements; • requirements for equipment to perform the measurements and analysis; • requirements for the measurement process; • requirements for the management of the metric calculation; • operation or run rules to configure, execute, and monitor the testing; • documentation and reporting requirements; • a validation process for the metric using the Deployed Power Assessment. The present document is not applicable to: • fully fault tolerant servers; • High Performance Computing (HPC) systems; • hyper-converged servers; • large scale servers; • servers with integrated APA(s); • networking equipment including network servers; • server appliances; • storage device including blade storage and storage servers. NOTE 2: Products whose feature set and intended operation are not addressed by active mode testing parameters are excluded from this evaluation method. The above list shows products for which SERT™ efficiency evaluations are not appropriate.

The present document does not address home servers and small servers that fall under the scope of mandate M/545 [i.8].

ETSI ETSI EN 303 470 V1.1.1 (2019-03) 9 2 References 2.1 Normative references References are either specific (identified by date of publication and/or edition number or version number) or non-specific. For specific references, only the cited version applies. For non-specific references, the latest version of the referenced document (including any amendments) applies. Referenced documents which are not found to be publicly available in the expected location might be found at https://docbox.etsi.org/Reference/. NOTE: While any hyperlinks included in this clause were valid at the time of publication, ETSI cannot guarantee their long term validity. The following referenced documents are necessary for the application of the present document. [1] CENELEC EN 62623:2013: "Desktop and notebook computers. Measurement of energy consumption". [2] Standard Performance Evaluation Corporation (SPEC): "Server Efficiency Rating Tool (SERT) version 2 Run and Reporting Rules". NOTE: Available at https://www.spec.org/sert2/SERT-runrules.pdf. [3] Standard Performance Evaluation Corporation (SPEC): "Server Efficiency Rating Tool (SERT) version 2 User Guide". NOTE: Available at https://www.spec.org/sert2/SERT-userguide.pdf. [4] IEEE 802.3™: "IEEE Standard for Ethernet". NOTE: Available at https://standards.ieee.org/findstds/standard/802.3-2015.html. [5] IEEE 802.3az™: "Energy Efficient Ethernet". 2.2 Informative references References are either specific (identified by date of publication and/or edition number or version number) or non-specific. For specific references, only the cited version applies. For non-specific references, the latest version of the referenced document (including any amendments) applies. NOTE: While any hyperlinks included in this clause were valid at the time of publication, ETSI cannot guarantee their long term validity. The following referenced documents are not necessary for the application of the present document but they assist the user with regard to a particular subject area. [i.1] CENELEC EN 60297 series: "Mechanical structures for electrical and electronic equipment. Dimensions of mechanical structures of the 482,6 mm (19 in) series". [i.2] ENERGY STAR®: "Product Specification for Computer Servers". [i.3] Standardization mandate M/462: "M/462 Standardisation mandate addressed to CEN, CENELEC and ETSI in the field of ICT to enable efficient energy use in fixed and mobile information and communication networks". [i.4] ETSI EN 300 119 series: "Equipment Engineering (EE); European telecommunication standard for equipment practice". [i.5] Standard Performance Evaluation Corporation (SPEC): "Server Efficiency Rating Tool (SERT) version 2 Design Document". NOTE: Available at https://www.spec.org/sert2/SERT-designdocument.pdf. SIST EN 303 470 V1.1.1:2019

ETSI ETSI EN 303 470 V1.1.1 (2019-03) 10 [i.6] SERT Client Configurations (JVM Options). NOTE: Available at https://www.spec.org/sert2/SERT-JVM_Options-2.0.html. [i.7] SERT Result File Fields. NOTE: Available at https://www.spec.org/sert2/SERT-resultfilefields.html. [i.8] Mandate M/545: "Commission Implementing Decision of 6.1.2016 on a standardisation request to the European standardisation organisations as regards computers and computer servers, in support of the implementation of Commission Regulation (EU) No 617/2013 of 26 June 2013, implementing Directive 2009/125/EC of the European Parliament and of the Council with regard to ecodesign requirements for computers and computer servers". 3 Definition of terms, symbols and abbreviations 3.1 Terms For the purposes of the present document, the following terms apply: a.c.-d.c. power supply unit: power supply unit that converts line-voltage alternating current (a.c.) input power into one or more direct current (d.c.) power outputs for powering a computer server active state: operational state of a computer server (as opposed to the idle state) in which the computer server is carrying out work in response to prior or concurrent external requests (e.g. instruction over the network) NOTE: The work includes, but is not restricted to, active processing and data seeking/retrieval from memory, cache, or internal/external storage while awaiting further input over the network. Auxiliary Processing Accelerator (APA): additional compute device installed in the computer server that handles parallelized workloads NOTE 1: This includes, but is not limited to, Graphical Processing Units (GPUs) or Field Programmable Gate Array chips which can be installed in a server either on Graphics or Extension add-in cards installed in general-purpose add-in expansion slots (e.g. GPGPUs, CPU accelerators, etc. installed in a PCI slot) or directly attached to a server component such as the motherboard.

NOTE 2: There are two specific types of APAs used in servers: a) Expansion APA: An APA that is on an add-in card installed in an add-in expansion slot (e.g. GPGPUs, CPU accelerators, etc. installed in a PCI slot). An expansion APA add-in card may include one or more APAs. b) Integrated APA: An APA that is integrated into the motherboard or CPU package or an expansion APA that has part of its subsystem, such as switches, included in the non-APA server configuration that would be used to run the energy efficiency test (SERTTM suite). blade chassis: enclosure that contains shared resources for the operation of blade servers, blade storage, and other blade form-factor devices NOTE: Shared resources provided by a chassis include, but are not restricted to, power supplies, data storage, and hardware for d.c. power distribution, thermal management, system management, and network services. blade server: computer server, designed for use in a blade chassis, that is a high-density device and functions as an independent computer server and includes at least one processor and system memory, which is dependent upon shared blade chassis resources (e.g. power supplies, cooling) for operation NOTE: A processor or memory module that is intended to scale up a standalone server is not considered a blade server. blade storage: storage device that is designed for use in a blade chassis and that is dependent upon shared blade chassis resources (e.g. power supplies, cooling) for operation SIST EN 303 470 V1.1.1:2019

ETSI ETSI EN 303 470 V1.1.1 (2019-03) 11 blade system: blade chassis and one or more removable blade servers and/or other units (e.g. blade storage, blade networking equipment) which provide a scalable means for combining multiple blade server or storage units in a single enclosure NOTE: A blade system is designed to allow service technicians to easily add or replace (hot-swap) blades in the field. buffered Double Data Rate (DDR) channel: channel or memory port connecting a memory controller to a defined number of memory devices (e.g. dual in-line memory modules (DIMMs)) in a computer server NOTE 1: A typical computer server may contain multiple memory controllers, which may in turn support one or more buffered DDR channels. NOTE 2:

Each buffered DDR channel serves only a fraction of the total addressable memory space in a computer server. computer server: computer, sold through enterprise channels, that provides services and manages networked resources for client devices

NOTE 1: Client devices include, but are not restricted to desktop computers, notebook computers, thin clients, wireless devices, Personal Digital Assistants, IP telephones, other computer servers, or other network devices. NOTE 2: A computer server is primarily accessed via network connections, versus directly-connected user input devices such as a keyboard or mouse. controller system: computer or computer server that manages a benchmark evaluation process data averaging interval: time period over which all samples captured by the high-speed sampling electronics of the power analyser are averaged to provide the measurement set d.c.-d.c. power supply unit: power supply unit that converts line-voltage direct current (d.c.) input power to one or more d.c. outputs for powering a computer server

NOTE: For purposes of the present document, a d.c-d.c. converter that is internal to a computer server and is used to convert a low voltage d.c. (e.g. 12 VDC) into other d.c. power o