Telecommunications and Internet converged Services and Protocols for Advanced Networking (TISPAN); NGN Congestion and Overload Control; Part 2: Core GOCAP and NOCA Entity Behaviours

DES/TISPAN-03034-2-NGN-R3

Zlite telekomunikacijske in internetne storitve ter protokoli za napredno omreženje (TISPAN) - Krmiljenje zadušitev in preobremenitev - 2. del: Obnašanje osrednjih delov GOCAP* in NOCA**

General Information

Status
Published
Publication Date
13-Jan-2010
Technical Committee
Current Stage
12 - Completion
Due Date
25-Jan-2010
Completion Date
14-Jan-2010
Standard
ETSI ES 283 039-2 V3.1.0 (2009-11) - Telecommunications and Internet converged Services and Protocols for Advanced Networking (TISPAN); NGN Congestion and Overload Control; Part 2: Core GOCAP and NOCA Entity Behaviours
English language
105 pages
sale 15% off
Preview
sale 15% off
Preview
Standard
ETSI ES 283 039-2 V3.1.1 (2010-01) - Telecommunications and Internet converged Services and Protocols for Advanced Networking (TISPAN); NGN Congestion and Overload Control; Part 2: Core GOCAP and NOCA Entity Behaviours
English language
105 pages
sale 15% off
Preview
sale 15% off
Preview

Standards Content (Sample)


Final draft ETSI ES 283 039-2 V3.1.0 (2009-11)
ETSI Standard
Telecommunications and Internet converged Services and
Protocols for Advanced Networking (TISPAN);
NGN Congestion and Overload Control;
Part 2: Core GOCAP and NOCA Entity Behaviours


2 Final draft ETSI ES 283 039-2 V3.1.0 (2009-11)

Reference
DES/TISPAN-03034-2-NGN-R3
Keywords
control, quality, protocol
ETSI
650 Route des Lucioles
F-06921 Sophia Antipolis Cedex - FRANCE

Tel.: +33 4 92 94 42 00  Fax: +33 4 93 65 47 16

Siret N° 348 623 562 00017 - NAF 742 C
Association à but non lucratif enregistrée à la
Sous-Préfecture de Grasse (06) N° 7803/88

Important notice
Individual copies of the present document can be downloaded from:
http://www.etsi.org
The present document may be made available in more than one electronic version or in print. In any case of existing or
perceived difference in contents between such versions, the reference version is the Portable Document Format (PDF).
In case of dispute, the reference shall be the printing on ETSI printers of the PDF version kept on a specific network drive
within ETSI Secretariat.
Users of the present document should be aware that the document may be subject to revision or change of status.
Information on the current status of this and other ETSI documents is available at
http://portal.etsi.org/tb/status/status.asp
If you find errors in the present document, please send your comment to one of the following services:
http://portal.etsi.org/chaircor/ETSI_support.asp
Copyright Notification
No part may be reproduced except as authorized by written permission.
The copyright and the foregoing restriction extend to reproduction in all media.

© European Telecommunications Standards Institute 2009.
All rights reserved.
TM TM TM TM
DECT , PLUGTESTS , UMTS , TIPHON , the TIPHON logo and the ETSI logo are Trade Marks of ETSI registered
for the benefit of its Members.
TM
3GPP is a Trade Mark of ETSI registered for the benefit of its Members and of the 3GPP Organizational Partners.
LTE™ is a Trade Mark of ETSI currently being registered
for the benefit of its Members and of the 3GPP Organizational Partners.
GSM® and the GSM logo are Trade Marks registered and owned by the GSM Association.
ETSI
3 Final draft ETSI ES 283 039-2 V3.1.0 (2009-11)
Contents
Intellectual Property Rights . 7
Foreword . 7
1 Scope . 8
2 References . 8
2.1 Normative references . 8
2.2 Informative references . 9
3 Definitions and abbreviations . 9
3.1 Definitions . 9
3.2 Abbreviations . 9
4 Control Architecture . 10
4.1 Description of NOCA Components . 10
4.2 Detailed Description of NOCA Components and Behaviour . 12
4.2.1 Overview . 12
4.2.2 Control Adaptor (CAProcess) . 14
4.2.2.1 Control Adaptor Data . 14
4.2.2.2 CAProcess signals . 15
4.2.2.3 Control Adaptor Behaviour . 15
4.2.2.4 Generating Control Adaptor Input . 19
4.2.3 Control Distribution (SDL: CDProcess) . 19
4.2.3.1 Control Distribution data . 19
4.2.3.2 Control Distribution Signals . 20
4.2.3.3 Control Distribution Behaviour . 20
4.2.4 CDRestriction . 23
4.2.4.1 CDRestriction Data . 24
4.2.4.2 CDRestrictor Signals . 24
4.2.4.3 CDRestrictor Behaviour . 25
4.2.5 Restrictor Manager (RMProcess) . 28
4.2.5.1 Restrictor Manager Data . 28
4.2.5.2 Restrictor Manager Signals . 29
4.2.5.3 Restrictor Manager Behaviour . 30
4.2.6 Restrictor . 33
4.2.6.1 Restrictor data . 33
4.2.6.2 Restrictor signals . 34
4.2.6.3 Restrictor behaviour . 34
4.2.7 GOCAP Transport . 35
4.2.7.1 The structure of the GOCAP transport layer. . 35
4.2.7.2 Channel Manager . 37
4.2.7.2.1 Channel Manager Data . 37
4.2.7.2.2 Channel Manager Signals . 37
4.2.7.2.3 Channel Manager Behaviour . 38
4.2.7.3 Shim Process . 39
4.2.7.3.1 Shim Process Signals . 39
4.2.7.3.2 Shim Process Behaviour . 39
4.2.7.4 GocapListener . 40
4.2.7.5 SessionHandler . 40
5 GOCAP over Diameter . 40
5.1 Introduction . 40
5.2 Use of the Diameter base protocol . 41
5.2.1 Advertising GOCAP support . 41
5.2.2 Securing Diameter messages . 41
5.2.3 Accounting functionality . 41
5.2.4 GOCAP commands. 41
5.2.4.1 AA-Request (AAR) command . 41
ETSI
4 Final draft ETSI ES 283 039-2 V3.1.0 (2009-11)
5.2.4.2 AA-Answer (AAA) Command . 42
5.2.4.3 Profile-Update-Request (PUR) command . 42
5.2.4.4 Profile-Update-Answer (PUA) command . 42
5.2.4.5 Session-Termination-Request (STR) command . 43
5.2.4.6 Session-Termination-Answer (STA) command . 43
5.2.4.7 Abort-Session-Request (ASR) command . 43
5.2.4.8 Abort-Session-Answer (ASA) command . 43
5.2.5 AVP definitions . 44
5.2.5.1 Auth_Scope . 44
5.2.5.2 AVP GOCAP-Body . 44
5.2.6 Restrictions on AVP values . 44
5.2.6.1 Auth-Request-Type . 44
5.2.6.2 Auth-Session-State AVP . 45
5.3 Procedures to be used with Diameter messages . 45
5.3.1 Introduction. 45
5.3.2 Diameter ChannelManager . 45
5.3.3 Diameter Shim . 46
5.3.3.1 Diameter Shim data . 46
5.3.3.2 Diameter shim behaviour . 46
5.3.3.3 Generating PUR messages . 49
5.3.4 Diameter Listener . 50
5.3.4.1 Diameter session initiation . 50
5.3.4.2 Diameter session termination . 51
5.3.4.3 Gocap commands . 51
5.3.5 Diameter Session Handler . 52
5.3.6 GOCAP Timers . 52
5.4 Diameter MSC charts . 53
5.4.1 Simple Diameter session . 53
6 GOCAP over SIP . 56
6.1 General . 56
6.2 Overview . 56
6.2.1 GOCAP Slave . 56
6.2.1.1 Subscription . 56
6.2.1.2 Receiving Notifications . 57
6.2.2 GOCAP Master . 57
6.2.2.1 Subscription . 57
6.2.2.2 Notification . 58
6.3 Detailed procedures . 58
6.3.1 Introduction. 58
6.3.2 GOCAP Master . 59
6.3.2.1 SIP ChannelManager . 59
6.3.2.2 SIP Shim . 60
6.3.2.2.1 SIP Shim data . 60
6.3.2.2.2 SIP shim behaviour . 60
6.3.2.2.3 Generating NOTIFY messages . 62
6.3.3 GOCAP slave. 63
6.3.3.1 SIP Listener . 63
6.3.3.1.1 SIP Session initiation . 64
6.3.3.1.2 Session termination . 65
6.3.3.1.3 Gocap commands . 65
6.3.3.2 SIP Session Handler . 65
Annex A (normative): ASN.1 data types and signal definitions . 67
A.1 ASN.1 definitions . 67
A.2 Signals . . 69
A.3 SDL description . 70
Annex B (normative): Congestion_Control event package . 71
B.1 Event Package Name . 71
ETSI
5 Final draft ETSI ES 283 039-2 V3.1.0 (2009-11)
B.2 Event Package Parameters . 71
B.3 SUBSCRIBE Bodies . 71
B.4 Subscription Duration. 71
B.5 NOTIFY Bodies . 71
B.6 Notifier Processing of SUBSCRIBE Requests. 71
B.7 Notifier Generation of NOTIFY Requests . . 71
B.8 Subscriber Processing of NOTIFY Requests . 72
B.9 Subscriber Generation of SUBSCRIBE Requests . 72
B.10 Handling of Forked Requests . 72
B.11 Rate of Notifications . 72
B.12 State Agents . 72
B.13 Use of URIs to Retrieve State . 72
Annex C (normative): XML Schema . . 73
C.1 Introduction . 73
C.2 XML Schema specification . 73
Annex D (informative): Generating System_state data . 77
D.1 Introduction . 77
D.2 Background . 77
D.3 Modelling CPU load . 78
D.4 Single processing system. 79
D.4.1 Arrival rate and Goal rate . 79
D.4.2 Scheduling the update . 80
D.4.3 Updating the arrival rate . 80
D.4.4 Updating the goal rate . 80
D.4.5 Variables. 82
D.4.6 Initialisation . 82
D.4.7 Configurable Parameters . 83
D.5 Multiple processing subsystems . 83
D.5.1 Scheduling the update . 85
D.5.2 Updating the arrival rate . 85
D.5.3 Updating the goal rate . 85
D.5.4 Special design considerations . 86
D.5.4.1 AS unavailability . 86
D.5.4.2 Late or missing updates . 86
Annex E (informative): Message Sequence Charts (Transport Independent) . 87
E.1 Adding sources . 87
E.1.1 Overview . 87
E.1.2 Data flows for addition of a source . 89
E.2 Deleting sources . 91
E.3 Overload onset and abatement . 92
E.3.1 Overview of overload onset and abatement . 92
E.3.2 Detailed view of data flows in overload . 94
E.4 Audit . 95
E.5 Switching to local restriction . 96
ETSI
6 Final draft ETSI ES 283 039-2 V3.1.0 (2009-11)
Annex F (informative): Adaptation behaviour discussion. 99
F.1 Adaptation algorithm behaviour . 99
F.2 Adaptation and control termination . 102
F.3 Capacity Modification Factor . 103
Annex G (informative): Bibliography . 104
History . 105

ETSI
7 Final draft ETSI ES 283 039-2 V3.1.0 (2009-11)
Intellectual Property Rights
IPRs essential or potentially essential to the present document may have been declared to ETSI. The information
pertaining to these essential IPRs, if any, is publicly available for ETSI members and non-members, and can be found
in ETSI SR 000 314: "Intellectual Property Rights (IPRs); Essential, or potentially Essential, IPRs notified to ETSI in
respect of ETSI standards", which is available from the ETSI Secretariat. Latest updates are available on the ETSI Web
server (http://webapp.etsi.org/IPR/home.asp).
Pursuant to the ETSI IPR Policy, no investigation, including IPR searches, has been carried out by ETSI. No guarantee
can be given as to the existence of other IPRs not referenced in ETSI SR 000 314 (or the updates on the ETSI Web
server) which are, or may be, or may become, essential to the present document.
Foreword
This ETSI Standard (ES) has been produced by ETSI Technical Committee Telecommunications and Internet
converged Services and Protocols for Advanced Networking (TISPAN), and is now submitted for the ETSI standards
Membership Approval Procedure.
The present document is part 2 of a multi-part deliverable covering NGN Overload and Congestion Control as identified
below:
Part 1: "Overview";
Part 2: "Core GOCAP and NOCA Entity Behaviours";
Part 3: "Overload and Congestion Control for H.248 MG/MGC";
Part 4: "Adaptative Control for the MGC";
Part 5: "ISDN overload control at the Access Gateway".
ETSI
8 Final draft ETSI ES 283 039-2 V3.1.0 (2009-11)
1 Scope
The present document describes the core features of the NGN Overload Control Architecture (NOCA) and the Generic
Overload Control Application Protocol (GOCAP). While it is usual for the architectural components to be specified
separately from the protocols that are used to communicate between them, the performance requirements of overload
controls are such that the coupling between architecture, protocol and implementation is very strong. This means that
the present document specifies the architecture, entity behaviours and protocol for the core NOCA/GOCAP together.
The way GOCAP and the NOCA entities are deployed to control traffic that uses a specific application protocol is
profiled via additional small shim specifications.
2 References
References are either specific (identified by date of publication and/or edition number or version number) or
non-specific.
• For a specific reference, subsequent revisions do not apply.
• Non-specific reference may be made only to a complete document or a part thereof and only in the following
cases:
- if it is accepted that it will be possible to use all future changes of the referenced document for the
purposes of the referring document;
- for informative references.
Referenced documents which are not found to be publicly available in the expected location might be found at
http://docbox.etsi.org/Reference.
NOTE: While any hyperlinks included in this clause were valid at the time of publication ETSI cannot guarantee
their long term validity.
2.1 Normative references
The following referenced documents are indispensable for the application of the present document. For dated
references, only the edition cited applies. For non-specific references, the latest edition of the referenced document
(including any amendments) applies.
[1] ETSI TS 182 018: "Telecommunications and Internet converged Services and Protocols for
Advanced Networking (TISPAN); Control of Processing Overload; Stage 2 Requirements".
[2] IETF RFC 3588: "Diameter Base Protocol".
[3] IETF RFC 4005: "Diameter Network Access Server Application".
[4] ETSI TS 133 210: "Digital cellular telecommunications system (Phase 2+); Universal Mobile
Telecommunications System (UMTS); LTE; 3G security; Network Domain Security (NDS); IP
network layer security (3GPP TS 33.210)".
[5] ETSI TS 129 329: "Digital cellular telecommunications system (Phase 2+); Universal Mobile
Telecommunications System (UMTS); LTE; Sh interface based on the Diameter protocol; Protocol
details (3GPP TS 29.329 version 8.4.0 Release 8)".
[6] IETF RFC 3265: "Session Initiation Protocol (SIP)-Specific Event Notification".
ETSI
9 Final draft ETSI ES 283 039-2 V3.1.0 (2009-11)
2.2 Informative references
The following referenced documents are not essential to the use of the present document but they assist the user with
regard to a particular subject area. For non-specific references, the latest version of the referenced document (including
any amendments) applies.
[i.1] ETSI ES 283 039-4: "Telecommunications and Internet converged Services and Protocols for
Advanced Networking (TISPAN); NGN Overload Control Architecture; Part 4: Adaptative
Control for the MGC".
[i.2] IETF RFC 4662: "A Session Initiation Protocol (SIP) Event Notification Extension for Resource
Lists".
3 Definitions and abbreviations
3.1 Definitions
For the purposes of the present document, the terms and definitions given in TS 182 018 [1] and the following apply:
application: software component(s) running on a system to provide service to end users or support the management of
the system
NOTE: In the present document the term application excludes those software components that implement the
NOCA.
application protocol: protocol used to enable application instances to communicate
control variable: time-varying parameter used to control actuators in a feedback loop, calculated on the basis of the
target and measured values of some system quantity
feedback loop: control mechanism where the result of changing an actuator is used ("fed back") into the algorithm used
to calculate future changes
load control: mechanism for controlling the workload of a system
overload: system workload exceeds a defined threshold of the processing capacity of that system
source: system that generates workload for another system
target: system that receives workload from another system
workload: amount of processing work a system has to perform
3.2 Abbreviations
For the purposes of the present document, the following abbreviations apply:
AAA AA-Answer
API Application Programming Interface
ASA Abort-Session-Answer
ASR Abort-Session-Request
AVP Attribute-Value Pair
CA Control Adaptor
CD Control Distribution
CDR CDRestriction
CEA Capabilities-Exchange-Answer
CER Capabilities-Exchange-Request
CM Channel Manager
FQDN Fully Qualified Domain Name
GOCAP Generic Overload Control Application Protocol
ETSI
10 Final draft ETSI ES 283 039-2 V3.1.0 (2009-11)
IP Internet Protocol
ISUP Integrated Service Digital Network User Part
NGN Next Generation Network
NOCA NGN Overload Control Architecture
PUA Profile-Update-Answer
PUR Profile-Update-Request
RM Restrictor Manager
SCTP Stream Control Transmission Protocol
SDL Specification and Description Language
SIP Session Initiation Protocol
SLA Service Level Agreement
STA Session-Termination-Answer
STR Session-Termination-Request
TCP Transmission Control Protocol
4 Control Architecture
4.1 Description of NOCA Components

Figure 1: Control components implementing a feedback control path
The NGN Overload Control Architecture (NOCA) aims to provide feedback based processing load control for hosts that
implement the functionality of NGN (and other) networks. Each feedback control loop indicated by the ovals in
Figure 1, comprises three key NOCA components, the Control Adaptor, Control Distribution and Restrictor. The
objective of the feedback loops is to enable the protected host to operate at the optimum load by restricting excess
workload originating from its nearest neighbours, so the restrictors are usually (though not always) located at those
nearest neighbours.
Control Adaptor:
The control adaptor receives data from the host system describing the current workload and system capacity
and uses this to derive a global leakrate which is passed to the control distribution. The control adaptor then
adjusts the global leak rate, in order that the current workload converges to a goal value equal to the system
capacity.
ETSI
11 Final draft ETSI ES 283 039-2 V3.1.0 (2009-11)
Control Distribution:
The control distribution component shares the global leak rate between the restrictors on the basis of simple
local policies. These policies enable defined service levels or fairness requirements to be realised. The control
distribution uses the Generic Overload Control Application Protocol (GOCAP) to transmit leak rate
information to restrictors on remote hosts.
Restrictor:
The restrictor is a leaky bucket rate limiter that is request priority aware. It is controlled by the leak rate
received from the control distribution component and restricts the workload presented to the host which the
control is protecting.
The components and functions shown in Figure 1 should be thought of as being located above the top level of the
protocol stacks running at the overloaded target (the GOCAP Master) and sources (GOCAP Slaves). As an example,
two communications application instances running on the source and target may use ISUP to communicate with each
other in establishing and clearing-down voice calls. Beneath ISUP, the protocol stack (in descending order) could be
SIP-I/TCP/IP.
In general, a host may be the source of excess processing load for other hosts as well as the target of excess processing
load from those other hosts. This means that a single host typically implements all the control components, together
with additional components to manage them. The components at a fully functional (from a GOCAP perspective) host
are shown in Figure 2.
GOCAP Enabled Node
Management
Function
Control
GOCAP
Distribution GOCAP
Control
Adaptor
Restrictor
Manager
LR
LR
Restrictor
Comms App/
Protocol
Protocol
Host OS
Figure 2: Structure of a GOCAP enabled host, arrows denote information flow - dashed arrows
represent replies to requests
Figure 2 introduces an additional NOCA component, the restrictor manager. This component is responsible for
co-ordinating multiple restrictors while providing a single interface to the host/application. All the NOCA components
are described in greater detail in the next clause.
Figure 2 also shows that internal interfaces are required between NOCA components and the network management
system for configuration, status enquiries, control statistics etc. Also shown are the interfaces between the host OS
and/or communications applications which are used for admission queries (used to reduce workload) and for the
protected system to send the data required to drive the control adaptor.
ETSI
12 Final draft ETSI ES 283 039-2 V3.1.0 (2009-11)
4.2 Detailed Description of NOCA Components and Behaviour
4.2.1 Overview
In the following clauses, the behaviours and attributes of the NOCA will be described in detail. To assist that
description, SDL diagrams are included. The signals and data types used in the SDL diagrams are defined in Annex A.
NOTE: The present document uses SDL to describe the required behaviour, but does not enforce a particular
method of realisation. SDL diagrams included in the present document are not meant to imply a particular
implementation in terms of processes/threads, message passing or function/procedure calls. In many cases
the signals given in the SDL diagrams may just be interpreted as function/procedure calls. The
implementer is free to realise this behaviour in the most appropriate and efficient way.
Figure 3 shows a representation in SDL of a GOCAP system with the explicit addition of a GOCAP signalling channel
between the GOCAP master and a remote slave (remote_gs). Figure 4 shows the key components of a GOCAP master -
a host that is protected from overload by restrictions on GOCAP slaves. Note that the Control Distribution function has
been split into two elements in Figure 4, the CDProcess and the CDRestriction. This is to help separate the description
of the leakrate calculation elements of the Control Distribution (in the CDProcess) from those aspects that relate to the
management of a particular restriction (the CDRestriction). Figure 5 gives a similar view of a GOCAP slave - a host
that implements restrictors to limit the flow of requests.
System GOCAP 1(1)
restrictor_status admit,reject admit,reject
master_app sys_mon slave_app
use GocapSignals;
use GocapTypes;
gocapMaster
request request
egate agate agate
gocapSlave
local_gs:gocapSlave remote_gs:gocapSlave
rgate rgate egate
audit new , new ,
set_rate, set_rate,
active_restrictions
halt halt, audit
restrictor_status system_state
add_src,
CDR_Id,
del_src,
lgate sgate restrictor_status
CDR_error
update_src
gm:gocapMaster
egate
comms_error,
mgmt
restrictor_status
mgate GSSAP
up, dow n,
use_channel,
restrictor_status
GMSAP
active_restrictions, open, close, new , active_restrictions,
restrictor_status, set_rate, halt restrictor_status
GSSM
comms_error get_audit
Gocap_Transport
GMSM
Figure 3: A representation of the GOCAP system using SDL
ETSI
13 Final draft ETSI ES 283 039-2 V3.1.0 (2009-11)
Block Type gocapMaster 1(1)
use GocapTypes;
use GocapSignals;
system_state system_state
CAProcess
/* Control Adaptor*/
system_mon
sgate
System load
update_origin information
There is a single CDProcess
for a GOCAP master
terminate,
glr_update
add_src, add_src,
CDR_Id, CDR_Id,
del_src, del_src,
CDR_error CDR_error
update_src
CDProcess update_src
/*Control Distribution*/
egate
There is one CDRestriction
process for each controlled
update_CDR,
source
halt_CDR,
new , new ,
delete_CDR
set_rate, set_rate,
restrictor_status halt restrictor_status halt
local
CDRestriction
lgate
up,dow n,
use_channel,
restrictor_status
To the local
restrictor manager
master
To the remote
restrictor manager
open, close, new ,
set_rate, halt
up,dow n,
use_channel,
mgate
restrictor_status
open, close, new ,
set_rate, halt
Figure 4: Signals and channels for a GOCAP Master. A node that hosts a
GOCAP Master will usually host a GOCAP Slave too
admit, reject
agate
request
Block Type gocapSlave 1(1)
admit,reject
/* Notes
This functionality is implemented on all
use GocapSignals;
GOCAP enabled sources.
app
*/
request
RMProcess
rmr
/* The restrictor
manager process */
Restrictor
ok,fail, update,
expired test,
confirm,
new ,set_rate,
delete
halt, audit
egate
slave
restrictor_error restrictor_error
active_restrictions
rgate
new , set_rate,
halt, audit
active_restrictions
Figure 5: A GOCAP Slave, showing restrictor management
ETSI
14 Final draft ETSI ES 283 039-2 V3.1.0 (2009-11)
4.2.2 Control Adaptor (CAProcess)
The purpose of the Control Adaptor is to master the adaptation of a control variable, which is translated into restriction
values by the distribution function, such that the arrival rate of requests associated with that Control Adaptor converges
on some goal arrival rate.
4.2.2.1 Control Adaptor Data
Each control adaptor shall be provisioned with the following parameters:
• the termination interval, terminationPending, used to identify when adaptive control should cease;
• control initiation factor, u, used to scale the initial value of the control variable;
• a minimum significant arrival rate change, d; and
• the effective origin scalar, a, used to modify the adaptation behaviour when the system capacity falls below the
sum of the capacity guarantees offered to the sources.
The use of the origin scalar parameter, a, is discussed in detail in Annex F.
Each control adaptor shall maintain the following state data:
• the control variable, C (also known as the global leakrate);
• the previous value of the control variable, oldC;
• a periodically-updated estimate of the total arrival rate of service requests, Y;
• the previous estimate of the arrival rate of service requests, oldY;
• periodically updated values for total arrival rate goal, G for service requests to that resource;
,
• the previous value of the total arrival rate goal, oldG;
• the current smallest s /w ratio multiplied by W, the sum of the weights, R, as signalled from the Control
i i
Distribution; and
• the current sum of static capacity allocations, S, as signalled from the Control Distribution.
The use of R and S is described in more detail in Annex F.
ETSI
15 Final draft ETSI ES 283 039-2 V3.1.0 (2009-11)
4.2.2.2 CAProcess signals
Table 1: Signals sent or received by the Control Adaptor Process
Signal Dirn Comments
system_state(Y Real, G Real) recv Received periodically by the CA process from thesystem. It informs
...


ETSI Standard
Telecommunications and Internet converged Services and
Protocols for Advanced Networking (TISPAN);
NGN Congestion and Overload Control;
Part 2: Core GOCAP and NOCA Entity Behaviours


2 ETSI ES 283 039-2 V3.1.1 (2010-01)

Reference
DES/TISPAN-03034-2-NGN-R3
Keywords
control, quality, protocol
ETSI
650 Route des Lucioles
F-06921 Sophia Antipolis Cedex - FRANCE

Tel.: +33 4 92 94 42 00  Fax: +33 4 93 65 47 16

Siret N° 348 623 562 00017 - NAF 742 C
Association à but non lucratif enregistrée à la
Sous-Préfecture de Grasse (06) N° 7803/88

Important notice
Individual copies of the present document can be downloaded from:
http://www.etsi.org
The present document may be made available in more than one electronic version or in print. In any case of existing or
perceived difference in contents between such versions, the reference version is the Portable Document Format (PDF).
In case of dispute, the reference shall be the printing on ETSI printers of the PDF version kept on a specific network drive
within ETSI Secretariat.
Users of the present document should be aware that the document may be subject to revision or change of status.
Information on the current status of this and other ETSI documents is available at
http://portal.etsi.org/tb/status/status.asp
If you find errors in the present document, please send your comment to one of the following services:
http://portal.etsi.org/chaircor/ETSI_support.asp
Copyright Notification
No part may be reproduced except as authorized by written permission.
The copyright and the foregoing restriction extend to reproduction in all media.

© European Telecommunications Standards Institute 2010.
All rights reserved.
TM TM TM TM
DECT , PLUGTESTS , UMTS , TIPHON , the TIPHON logo and the ETSI logo are Trade Marks of ETSI registered
for the benefit of its Members.
TM
3GPP is a Trade Mark of ETSI registered for the benefit of its Members and of the 3GPP Organizational Partners.
LTE™ is a Trade Mark of ETSI currently being registered
for the benefit of its Members and of the 3GPP Organizational Partners.
GSM® and the GSM logo are Trade Marks registered and owned by the GSM Association.
ETSI
3 ETSI ES 283 039-2 V3.1.1 (2010-01)
Contents
Intellectual Property Rights . 7
Foreword . 7
1 Scope . 8
2 References . 8
2.1 Normative references . 8
2.2 Informative references . 9
3 Definitions and abbreviations . 9
3.1 Definitions . 9
3.2 Abbreviations . 9
4 Control Architecture . 10
4.1 Description of NOCA Components . 10
4.2 Detailed Description of NOCA Components and Behaviour . 12
4.2.1 Overview . 12
4.2.2 Control Adaptor (CAProcess) . 14
4.2.2.1 Control Adaptor Data . 14
4.2.2.2 CAProcess signals . 15
4.2.2.3 Control Adaptor Behaviour . 15
4.2.2.4 Generating Control Adaptor Input . 19
4.2.3 Control Distribution (SDL: CDProcess) . 19
4.2.3.1 Control Distribution data . 19
4.2.3.2 Control Distribution Signals . 20
4.2.3.3 Control Distribution Behaviour . 20
4.2.4 CDRestriction . 23
4.2.4.1 CDRestriction Data . 24
4.2.4.2 CDRestrictor Signals . 24
4.2.4.3 CDRestrictor Behaviour . 25
4.2.5 Restrictor Manager (RMProcess) . 28
4.2.5.1 Restrictor Manager Data . 29
4.2.5.2 Restrictor Manager Signals . 30
4.2.5.3 Restrictor Manager Behaviour . 30
4.2.6 Restrictor . 33
4.2.6.1 Restrictor data . 34
4.2.6.2 Restrictor signals . 34
4.2.6.3 Restrictor behaviour . 34
4.2.7 GOCAP Transport . 35
4.2.7.1 The structure of the GOCAP transport layer. . 35
4.2.7.2 Channel Manager . 37
4.2.7.2.1 Channel Manager Data . 37
4.2.7.2.2 Channel Manager Signals . 37
4.2.7.2.3 Channel Manager Behaviour . 38
4.2.7.3 Shim Process . 39
4.2.7.3.1 Shim Process Signals . 39
4.2.7.3.2 Shim Process Behaviour . 39
4.2.7.4 GocapListener . 40
4.2.7.5 SessionHandler . 40
5 GOCAP over Diameter . 40
5.1 Introduction . 40
5.2 Use of the Diameter base protocol . 41
5.2.1 Advertising GOCAP support . 41
5.2.2 Securing Diameter messages . 41
5.2.3 Accounting functionality . 41
5.2.4 GOCAP commands. 41
5.2.4.1 AA-Request (AAR) command . 41
ETSI
4 ETSI ES 283 039-2 V3.1.1 (2010-01)
5.2.4.2 AA-Answer (AAA) Command . 42
5.2.4.3 Profile-Update-Request (PUR) command . 42
5.2.4.4 Profile-Update-Answer (PUA) command . 43
5.2.4.5 Session-Termination-Request (STR) command . 43
5.2.4.6 Session-Termination-Answer (STA) command . 43
5.2.4.7 Abort-Session-Request (ASR) command . 44
5.2.4.8 Abort-Session-Answer (ASA) command . 44
5.2.5 AVP definitions . 44
5.2.5.1 Auth_Scope . 45
5.2.5.2 AVP GOCAP-Body . 45
5.2.6 Restrictions on AVP values . 45
5.2.6.1 Auth-Request-Type . 45
5.2.6.2 Auth-Session-State AVP . 45
5.3 Procedures to be used with Diameter messages . 45
5.3.1 Introduction. 45
5.3.2 Diameter ChannelManager . 46
5.3.3 Diameter Shim . 46
5.3.3.1 Diameter Shim data . 47
5.3.3.2 Diameter shim behaviour . 47
5.3.3.3 Generating PUR messages . 49
5.3.4 Diameter Listener . 50
5.3.4.1 Diameter session initiation . 51
5.3.4.2 Diameter session termination . 52
5.3.4.3 Gocap commands . 52
5.3.5 Diameter Session Handler . 52
5.3.6 GOCAP Timers . 54
5.4 Diameter MSC charts . 54
5.4.1 Simple Diameter session . 54
6 GOCAP over SIP . 56
6.1 General . 56
6.2 Overview . 57
6.2.1 GOCAP Slave . 57
6.2.1.1 Subscription . 57
6.2.1.2 Receiving Notifications . 57
6.2.2 GOCAP Master . 58
6.2.2.1 Subscription . 58
6.2.2.2 Notification . 58
6.3 Detailed procedures . 59
6.3.1 Introduction. 59
6.3.2 GOCAP Master . 59
6.3.2.1 SIP ChannelManager . 59
6.3.2.2 SIP Shim . 60
6.3.2.2.1 SIP Shim data . 60
6.3.2.2.2 SIP shim behaviour . 60
6.3.2.2.3 Generating NOTIFY messages . 62
6.3.3 GOCAP slave. 63
6.3.3.1 SIP Listener . 63
6.3.3.1.1 SIP Session initiation . 64
6.3.3.1.2 Session termination . 65
6.3.3.1.3 Gocap commands . 65
6.3.3.2 SIP Session Handler . 65
Annex A (normative): ASN.1 data types and signal definitions . 67
A.1 ASN.1 definitions . 67
A.2 Signals . 69
A.3 SDL description . 70
Annex B (normative): Congestion_Control event package . 71
B.1 Event Package Name . 71
ETSI
5 ETSI ES 283 039-2 V3.1.1 (2010-01)
B.2 Event Package Parameters . 71
B.3 SUBSCRIBE Bodies . 71
B.4 Subscription Duration. 71
B.5 NOTIFY Bodies . 71
B.6 Notifier Processing of SUBSCRIBE Requests. 71
B.7 Notifier Generation of NOTIFY Requests . . 71
B.8 Subscriber Processing of NOTIFY Requests . 72
B.9 Subscriber Generation of SUBSCRIBE Requests . 72
B.10 Handling of Forked Requests . 72
B.11 Rate of Notifications . 72
B.12 State Agents . 72
B.13 Use of URIs to Retrieve State . 72
Annex C (normative): XML Schema . . 73
C.1 Introduction . 73
C.2 XML Schema specification . 73
Annex D (informative): Generating System_state data . 77
D.1 Introduction . 77
D.2 Background . 77
D.3 Modelling CPU load . 78
D.4 Single processing system. 79
D.4.1 Arrival rate and Goal rate . 79
D.4.2 Scheduling the update . 80
D.4.3 Updating the arrival rate . 80
D.4.4 Updating the goal rate . 80
D.4.5 Variables. 82
D.4.6 Initialisation . 82
D.4.7 Configurable Parameters . 83
D.5 Multiple processing subsystems . 83
D.5.1 Scheduling the update . 85
D.5.2 Updating the arrival rate . 85
D.5.3 Updating the goal rate . 85
D.5.4 Special design considerations . 86
D.5.4.1 AS unavailability . 86
D.5.4.2 Late or missing updates . 86
Annex E (informative): Message Sequence Charts (Transport Independent) . 87
E.1 Adding sources . 87
E.1.1 Overview . 87
E.1.2 Data flows for addition of a source . 89
E.2 Deleting sources . 91
E.3 Overload onset and abatement . 92
E.3.1 Overview of overload onset and abatement . 92
E.3.2 Detailed view of data flows in overload . 94
E.4 Audit . 95
E.5 Switching to local restriction . 96
ETSI
6 ETSI ES 283 039-2 V3.1.1 (2010-01)
Annex F (informative): Adaptation behaviour discussion. 99
F.1 Adaptation algorithm behaviour . 99
F.2 Adaptation and control termination . 102
F.3 Capacity Modification Factor . 103
Annex G (informative): Bibliography . 104
History . 105

ETSI
7 ETSI ES 283 039-2 V3.1.1 (2010-01)
Intellectual Property Rights
IPRs essential or potentially essential to the present document may have been declared to ETSI. The information
pertaining to these essential IPRs, if any, is publicly available for ETSI members and non-members, and can be found
in ETSI SR 000 314: "Intellectual Property Rights (IPRs); Essential, or potentially Essential, IPRs notified to ETSI in
respect of ETSI standards", which is available from the ETSI Secretariat. Latest updates are available on the ETSI Web
server (http://webapp.etsi.org/IPR/home.asp).
Pursuant to the ETSI IPR Policy, no investigation, including IPR searches, has been carried out by ETSI. No guarantee
can be given as to the existence of other IPRs not referenced in ETSI SR 000 314 (or the updates on the ETSI Web
server) which are, or may be, or may become, essential to the present document.
Foreword
This ETSI Standard (ES) has been produced by ETSI Technical Committee Telecommunications and Internet
converged Services and Protocols for Advanced Networking (TISPAN).
The present document is part 2 of a multi-part deliverable covering NGN Overload and Congestion Control as identified
below:
Part 1: "Overview";
Part 2: "Core GOCAP and NOCA Entity Behaviours";
Part 3: "Overload and Congestion Control for H.248 MG/MGC";
Part 4: "Adaptative Control for the MGC";
Part 5: "ISDN overload control at the Access Gateway".
ETSI
8 ETSI ES 283 039-2 V3.1.1 (2010-01)
1 Scope
The present document describes the core features of the NGN Overload Control Architecture (NOCA) and the Generic
Overload Control Application Protocol (GOCAP). While it is usual for the architectural components to be specified
separately from the protocols that are used to communicate between them, the performance requirements of overload
controls are such that the coupling between architecture, protocol and implementation is very strong. This means that
the present document specifies the architecture, entity behaviours and protocol for the core NOCA/GOCAP together.
The way GOCAP and the NOCA entities are deployed to control traffic that uses a specific application protocol is
profiled via additional small shim specifications.
2 References
References are either specific (identified by date of publication and/or edition number or version number) or
non-specific.
• For a specific reference, subsequent revisions do not apply.
• Non-specific reference may be made only to a complete document or a part thereof and only in the following
cases:
- if it is accepted that it will be possible to use all future changes of the referenced document for the
purposes of the referring document;
- for informative references.
Referenced documents which are not found to be publicly available in the expected location might be found at
http://docbox.etsi.org/Reference.
NOTE: While any hyperlinks included in this clause were valid at the time of publication ETSI cannot guarantee
their long term validity.
2.1 Normative references
The following referenced documents are indispensable for the application of the present document. For dated
references, only the edition cited applies. For non-specific references, the latest edition of the referenced document
(including any amendments) applies.
[1] ETSI TS 182 018: "Telecommunications and Internet converged Services and Protocols for
Advanced Networking (TISPAN); Control of Processing Overload; Stage 2 Requirements".
[2] IETF RFC 3588: "Diameter Base Protocol".
[3] IETF RFC 4005: "Diameter Network Access Server Application".
[4] ETSI TS 133 210: "Digital cellular telecommunications system (Phase 2+); Universal Mobile
Telecommunications System (UMTS); LTE; 3G security; Network Domain Security (NDS); IP
network layer security (3GPP TS 33.210)".
[5] ETSI TS 129 329: "Digital cellular telecommunications system (Phase 2+); Universal Mobile
Telecommunications System (UMTS); LTE; Sh interface based on the Diameter protocol; Protocol
details (3GPP TS 29.329 version 8.4.0 Release 8)".
[6] IETF RFC 3265: "Session Initiation Protocol (SIP)-Specific Event Notification".
ETSI
9 ETSI ES 283 039-2 V3.1.1 (2010-01)
2.2 Informative references
The following referenced documents are not essential to the use of the present document but they assist the user with
regard to a particular subject area. For non-specific references, the latest version of the referenced document (including
any amendments) applies.
[i.1] ETSI ES 283 039-4: "Telecommunications and Internet converged Services and Protocols for
Advanced Networking (TISPAN); NGN Overload Control Architecture; Part 4: Adaptative
Control for the MGC".
[i.2] IETF RFC 4662: "A Session Initiation Protocol (SIP) Event Notification Extension for Resource
Lists".
3 Definitions and abbreviations
3.1 Definitions
For the purposes of the present document, the terms and definitions given in TS 182 018 [1] and the following apply:
application: software component(s) running on a system to provide service to end users or support the management of
the system
NOTE: In the present document the term application excludes those software components that implement the
NOCA.
application protocol: protocol used to enable application instances to communicate
control variable: time-varying parameter used to control actuators in a feedback loop, calculated on the basis of the
target and measured values of some system quantity
feedback loop: control mechanism where the result of changing an actuator is used ("fed back") into the algorithm used
to calculate future changes
load control: mechanism for controlling the workload of a system
overload: system workload exceeds a defined threshold of the processing capacity of that system
source: system that generates workload for another system
target: system that receives workload from another system
workload: amount of processing work a system has to perform
3.2 Abbreviations
For the purposes of the present document, the following abbreviations apply:
AAA AA-Answer
API Application Programming Interface
ASA Abort-Session-Answer
ASR Abort-Session-Request
AVP Attribute-Value Pair
CA Control Adaptor
CD Control Distribution
CDR CDRestriction
CEA Capabilities-Exchange-Answer
CER Capabilities-Exchange-Request
CM Channel Manager
FQDN Fully Qualified Domain Name
GOCAP Generic Overload Control Application Protocol
ETSI
10 ETSI ES 283 039-2 V3.1.1 (2010-01)
IP Internet Protocol
ISUP Integrated Service Digital Network User Part
NGN Next Generation Network
NOCA NGN Overload Control Architecture
PUA Profile-Update-Answer
PUR Profile-Update-Request
RM Restrictor Manager
SCTP Stream Control Transmission Protocol
SDL Specification and Description Language
SIP Session Initiation Protocol
SLA Service Level Agreement
STA Session-Termination-Answer
STR Session-Termination-Request
TCP Transmission Control Protocol
4 Control Architecture
4.1 Description of NOCA Components

Figure 1: Control components implementing a feedback control path
The NGN Overload Control Architecture (NOCA) aims to provide feedback based processing load control for hosts that
implement the functionality of NGN (and other) networks. Each feedback control loop indicated by the ovals in
Figure 1, comprises three key NOCA components, the Control Adaptor, Control Distribution and Restrictor. The
objective of the feedback loops is to enable the protected host to operate at the optimum load by restricting excess
workload originating from its nearest neighbours, so the restrictors are usually (though not always) located at those
nearest neighbours.
Control Adaptor:
The control adaptor receives data from the host system describing the current workload and system capacity
and uses this to derive a global leakrate which is passed to the control distribution. The control adaptor then
adjusts the global leak rate, in order that the current workload converges to a goal value equal to the system
capacity.
ETSI
11 ETSI ES 283 039-2 V3.1.1 (2010-01)
Control Distribution:
The control distribution component shares the global leak rate between the restrictors on the basis of simple
local policies. These policies enable defined service levels or fairness requirements to be realised. The control
distribution uses the Generic Overload Control Application Protocol (GOCAP) to transmit leak rate
information to restrictors on remote hosts.
Restrictor:
The restrictor is a leaky bucket rate limiter that is request priority aware. It is controlled by the leak rate
received from the control distribution component and restricts the workload presented to the host which the
control is protecting.
The components and functions shown in Figure 1 should be thought of as being located above the top level of the
protocol stacks running at the overloaded target (the GOCAP Master) and sources (GOCAP Slaves). As an example,
two communications application instances running on the source and target may use ISUP to communicate with each
other in establishing and clearing-down voice calls. Beneath ISUP, the protocol stack (in descending order) could be
SIP-I/TCP/IP.
In general, a host may be the source of excess processing load for other hosts as well as the target of excess processing
load from those other hosts. This means that a single host typically implements all the control components, together
with additional components to manage them. The components at a fully functional (from a GOCAP perspective) host
are shown in Figure 2.
GOCAP Enabled Node
Management
Function
Control
GOCAP
Distribution GOCAP
Control
Adaptor
Restrictor
Manager
LR
LR
Restrictor
Comms App/
Protocol
Protocol
Host OS
Figure 2: Structure of a GOCAP enabled host, arrows denote information flow - dashed arrows
represent replies to requests
Figure 2 introduces an additional NOCA component, the restrictor manager. This component is responsible for
co-ordinating multiple restrictors while providing a single interface to the host/application. All the NOCA components
are described in greater detail in the next clause.
Figure 2 also shows that internal interfaces are required between NOCA components and the network management
system for configuration, status enquiries, control statistics etc. Also shown are the interfaces between the host OS
and/or communications applications which are used for admission queries (used to reduce workload) and for the
protected system to send the data required to drive the control adaptor.
ETSI
12 ETSI ES 283 039-2 V3.1.1 (2010-01)
4.2 Detailed Description of NOCA Components and Behaviour
4.2.1 Overview
In the following clauses, the behaviours and attributes of the NOCA will be described in detail. To assist that
description, SDL diagrams are included. The signals and data types used in the SDL diagrams are defined in Annex A.
NOTE: The present document uses SDL to describe the required behaviour, but does not enforce a particular
method of realisation. SDL diagrams included in the present document are not meant to imply a particular
implementation in terms of processes/threads, message passing or function/procedure calls. In many cases
the signals given in the SDL diagrams may just be interpreted as function/procedure calls. The
implementer is free to realise this behaviour in the most appropriate and efficient way.
Figure 3 shows a representation in SDL of a GOCAP system with the explicit addition of a GOCAP signalling channel
between the GOCAP master and a remote slave (remote_gs). Figure 4 shows the key components of a GOCAP master -
a host that is protected from overload by restrictions on GOCAP slaves. Note that the Control Distribution function has
been split into two elements in Figure 4, the CDProcess and the CDRestriction. This is to help separate the description
of the leakrate calculation elements of the Control Distribution (in the CDProcess) from those aspects that relate to the
management of a particular restriction (the CDRestriction). Figure 5 gives a similar view of a GOCAP slave - a host
that implements restrictors to limit the flow of requests.
System GOCAP 1(1)
restrictor_status admit,reject admit,reject
master_app sys_mon slave_app
use GocapSignals;
use GocapTypes;
gocapMaster
request request
egate agate agate
gocapSlave
local_gs:gocapSlave remote_gs:gocapSlave
rgate rgate egate
audit new , new ,
set_rate, set_rate,
active_restrictions
halt halt, audit
restrictor_status system_state
add_src,
CDR_Id,
del_src,
lgate sgate restrictor_status
CDR_error
update_src
gm:gocapMaster
egate
comms_error,
mgmt
restrictor_status
mgate GSSAP
up, dow n,
use_channel,
restrictor_status
GMSAP
active_restrictions, open, close, new , active_restrictions,
restrictor_status, set_rate, halt restrictor_status
GSSM
comms_error get_audit
Gocap_Transport
GMSM
Figure 3: A representation of the GOCAP system using SDL
ETSI
13 ETSI ES 283 039-2 V3.1.1 (2010-01)
Block Type gocapMaster 1(1)
use GocapTypes;
use GocapSignals;
system_state system_state
CAProcess
/* Control Adaptor*/
system_mon
sgate
System load
update_origin information
There is a single CDProcess
for a GOCAP master
terminate,
glr_update
add_src, add_src,
CDR_Id, CDR_Id,
del_src, del_src,
CDR_error CDR_error
update_src
CDProcess update_src
/*Control Distribution*/
egate
There is one CDRestriction
process for each controlled
update_CDR,
source
halt_CDR,
new , new ,
delete_CDR
set_rate, set_rate,
restrictor_status halt restrictor_status halt
local
CDRestriction
lgate
up,dow n,
use_channel,
restrictor_status
To the local
restrictor manager
master
To the remote
restrictor manager
open, close, new ,
set_rate, halt
up,dow n,
use_channel,
mgate
restrictor_status
open, close, new ,
set_rate, halt
Figure 4: Signals and channels for a GOCAP Master. A node that hosts a
GOCAP Master will usually host a GOCAP Slave too
admit, reject
agate
request
Block Type gocapSlave 1(1)
admit,reject
/* Notes
This functionality is implemented on all
use GocapSignals;
GOCAP enabled sources.
app
*/
request
RMProcess
rmr
/* The restrictor
manager process */
Restrictor
ok,fail, update,
expired test,
confirm,
new ,set_rate,
delete
halt, audit
egate
slave
restrictor_error restrictor_error
active_restrictions
rgate
new , set_rate,
halt, audit
active_restrictions
Figure 5: A GOCAP Slave, showing restrictor management
ETSI
14 ETSI ES 283 039-2 V3.1.1 (2010-01)
4.2.2 Control Adaptor (CAProcess)
The purpose of the Control Adaptor is to master the adaptation of a control variable, which is translated into restriction
values by the distribution function, such that the arrival rate of requests associated with that Control Adaptor converges
on some goal arrival rate.
4.2.2.1 Control Adaptor Data
Each control adaptor shall be provisioned with the following parameters:
• the termination interval, terminationPending, used to identify when adaptive control should cease;
• control initiation factor, u, used to scale the initial value of the control variable;
• a minimum significant arrival rate change, d; and
• the effective origin scalar, a, used to modify the adaptation behaviour when the system capacity falls below the
sum of the capacity guarantees offered to the sources.
The use of the origin scalar parameter, a, is discussed in detail in Annex F.
Each control adaptor shall maintain the following state data:
• the control variable, C (also known as the global leakrate);
• the previous value of the control variable, oldC;
• a periodically-updated estimate of the total arrival rate of service requests, Y;
• the previous estimate of the arrival rate of service requests, oldY;
• periodically updated values for total arrival rate goal, G for service requests to that resource;
,
• the previous value of the total arrival rate goal, oldG;
• the current smallest s /w ratio multiplied by W, the sum of the weights, R, as signalled from the Control
i i
Distribution; and
• the current sum of static capacity allocations, S, as signalled from the Control Distribution.
The use of R and S is described in more detail in Annex F.
ETSI
15 ETSI ES 283 039-2 V3.1.1 (2010-01)
4.2.2.2 CAProcess signals
Table 1: Signals sent or received by the Control Adaptor Process
Signal Dirn Comments
system_state(Y Real, G Real) recv Received periodically by the CA process from thesystem. It informs the CA of
the current work arrival rate, Y, and the goal work arrival rate (or the system
capacity), G, and effectively drives the control adaptation. For simple systems
where the host is servicing only one request type, the work arrival rate may
simply be the rate at whic
...

Questions, Comments and Discussion

Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.

Loading comments...