Logging EcoNAT Documentation / Quick system start / Logging
- Subscriber’s connection log settings
- System logging setup
- Quality of Experience
- Logging subscribers requests to web servers
Subscriber’s connection log settings
It is required by the legislation of some countries that all the information about the allocation of IP-addresses and/or port or block of ports must be saved. For maintaining this feature EcoNAT uses the syslog protocol as standard mechanism of logging.
In the branch system connection_log you can set the settings for the connection log. To start logging you should set the parameter enable.
In the case of using the platform with multiple network interfaces allocated for connection log, these interfaces are combined in a static virtual channel through which one are sent log packets. For the platforms with a single log interface virtual static channel is set on a single interface. In both cases for virtual channel will be assigned a synthetic IP-address of the source, so when you try to run ping command on this address, ICMP requests will remain unanswered (except of logging throught the mng interface case). Log packets will be sent by all connected network logging interfaces using Round-Robin algorithm.
The names of the network interfaces for logging are specified in section Hardware.
Connection_log basic parameters are described in a table below.
Parameter | Description |
---|---|
enable or disable | Enable or disable connection logging |
log_servers | Syslog servers addresses and ports for which logging will be carried out (logging would proceed in parallel on all available servers from the list, that is, each server will obtain information about all connections). Currently, the maximum number of servers is limited to two |
log_interface | The interface through which logging will be carried out. Possible values: default - through the logging interface, mng - via mng interface |
ip_address | Source IP-address and subnet mask (use '/') of the virtual channel in which logging network interfaces are merged |
mac | Source MAC-address of the virtual channel in which logging network interfaces are merged (if not specified, the MAC-address one of a network interfaces will be set) |
gateway | Default gateway for the virtual channel in which logging network interfaces are merged. Required if not all syslog server specified in log_servers parameter are on the subnet specified in the parameter ip_address |
strip_tags | In the mirroring mode, EcoNAT sends a connection interrupt packet (for HTTPS) or a redirection packet (for HTTP) to the subscriber via the network interface. When receiving tagged traffic and when the parameter is on, the tag (or double-tag) is cut off. When the parameter is off, the redirect or interrupt packet is sent to the logging network interface with the same parameters of the processed traffic |
that_mac | Syslog server MAC address in the log_servers section for the nearest L3 neighbour. If the option is not set, the MAC-address is calculated by the ARP protocol. Using this option reduces the chance of data loss at the start of logging provided a large load. EcoNAT able to process and log the more than 5 million connections per second at a full load. If a syslog server will respond to the ARP request, for example in 10 ms, in the queue may accumulate 50,000 connections waiting to be sent |
timeskew | Shift of the time indicated in the logs relative to Greenwich. Set in minutes. For example, for Moscow, the parameter value must be 180 |
pack_msgs | Enables packaging several reports in one message. This reduces the size of the logs and the network load |
Logging modes. Syslog logging
Ports for address translation for subscribers in CGNAT mode are allocated in blocks of 128 ports at a time. The next block is issued only when the exhaustion ports occurs in the previous block. Due to the block allocation, you may reduce volume of logs, as with the proper settings, instead of the number of reports on the allocation of ports to subscribers, will only one allocating the range of 128 ports (block) message.
EcoNAT supports multiple formats logging. The following describes the appropriate settings connection_log when logging in the syslog format.
Parameter | Description |
---|---|
log_format | Parameter indicates the type of logging:
|
log_on_release | The parameter indicates whether or not to send a message to connection_log if translation or block is released. The message is always sent when translation or block is created. If log_individual_conn enabled, a message is generated when occurs the release of each translation, otherwise – in case of releasing the block |
log_individual_conn | Parameter specifies whether to log individual connections, or you may log blocks of ports only |
use_hex_format | Allows to use hexadecimal format for the output log, which reduces the size of the log, while preserving the informational component. If disabled, the fixed decimal format is used, for example:010.210.000.012:00080 |
pack_msgs | Enables packaging several reports in one syslog message. This reduces the size of the logs and the network load |
facility | For the generated syslog messages sets a category of entity that generates the message for further processing and filtering. Possible values are from 16 to 23. These values relevant to the codes in the RFC 5424, indicating the subjects of local origin (local use 0 (local0) local use 7 (local7)). The default value – 16 |
severity | For the generated syslog messages sets severity for the convenience of further processing and filtering. Possible values are from 0 to 7, recommended – from 5 to 7. These values relevant to the codes in the RFC 5424, indicating the importance levels of messages:
|
The main modes of connection logging and recommended settings are presented in the table below .
The ratio of size/ readability of logs | log_on_release | log_individual_conn | use_hex_format | pack_msg s |
---|---|---|---|---|
The minimum log size (Ports blocks) | No | No | Yes | Yes |
The small size of the log, but it is more readable | No | No | No | No |
The small size of the log (connections) | No | Yes | Yes | Yes |
More readable logs (connections) | Yes | Yes | No | Yes |
Debug mode (most readable logs, but the large size) | Yes | Yes | No | No |
- If the providers logging storage system is well established (that is, everything is logged and stored without losses), then it is recommended to set for the four above parameters, the value No.
- If there are losses in the providers logging system, it makes sense to enable the log_on_release. Then, in case of loss of the opening of the connection message will be additionally sent the message about the closure, which will reduce the probability of loss of message.
MyEcoNAT:1:# root
MyEcoNAT:2:# system connection_log
MyEcoNAT:3:system.connection_log# log_servers ( 10.0.22.78:514 )
MyEcoNAT:4:system.connection_log# ip_address 10.0.22.33/255.255.255.0
MyEcoNAT:5:system.connection_log# log_on_release on
MyEcoNAT:6:system.connection_log# log_individual_conn on
MyEcoNAT:7:system.connection_log# pack_msgs off
MyEcoNAT:8:system.connection_log# enable
The syslog logging format: <Syslog server date time > < EcoNAT IP address> <EcoNAT date time> <EcoNAT name> | <Destination IP address (DST)>:<Port> < IP address to which the translation is done >:<Port> <Source IP address (SRC)> <Protocol identifier>.
Example:
Mar 3 14:36:58 10.210.1.234 2016-03-03T11:39:55+00:03 eco101 | 192.168.008.008:01024 A 060.000.000.226:01024 E 010.000.003.254:01024 UDPIP addresses are recorded in the three-digit format, for example, the address of 10.1.1.200 will be presented as 010.001.001.200.Below are a few examples of the log format settings. For convenience, some of the lines before the vertical bar is not shown.
Port blocks logging by packaging multiple network event messages to one syslog message. In this case, the log includes the NAT translation address, with used block of ports and IP address of the source.
log_on_release off
| 060.000.000.020:01024-01278 EA 010.000.003.250 UDP 060.000.000.018:01024-01278 EA 010.000.001.251 UDP 060.000.000.017:01024-01278 EA 010.000.002.251 UDP 060.000.000.015:01024-01278 EA 010.000.000.252 UDP 060.000.000.012:01024-01278 EA 010.000.003.252 UDP 060.000.000.010:01024-01278 EA 010.000.001.253 UDP 060.000.000.009:01024-01278 EA 010.000.002.253 UDP 060.000.000.007:01024-01278 EA 010.000.000.254 UDP 060.000.000.004:01024-01278 EA 010.000.003.254 UDP 060.000.000.002:01024-01278 EA 010.000.001.255 UDP 060.000.000.001:01024-01278 EA 010.000.002.255 UDP
Logging of each connection with the packaging of multiple network event messages to one syslog message. In this case, the log includes three addresses (destination, translation, source) specifying the port. Several events are packed into a single message.
Settings:
| 192.168.008.008:01024 A 060.000.000.006:01024 E 010.000.001.254:01024 UDP 192.168.008.008:01024 A 060.000.000.005:01024 E 010.000.002.254:01024 UDP 192.168.008.008:01024 A 060.000.000.003:01024 E 010.000.000.255:01024 UDP 192.168.008.008:01024 A 060.000.000.000:01024 E 010.000.003.255:01024 UDP
| 192.168.008.008:01024 A 060.000.000.010:01024 E 010.000.001.253:01024 UDP 192.168.008.008:01024 A 060.000.000.009:01024 E 010.000.002.253:01024 UDP 192.168.008.008:01024 A 060.000.000.007:01024 E 010.000.000.254:01024 UDP 192.168.008.008:01024 A 060.000.000.004:01024 E 010.000.003.254:01024 UDP 192.168.008.008:01024 A 060.000.000.002:01024 E 010.000.001.255:01024 UDP 192.168.008.008:01024 A 060.000.000.001:01024 E 010.000.002.255:01024 UDP
Logging of each connection without packaging. In this case, the log includes all three addresses (destination, translation, source) specifying the port. For each event a new message is created.
Settings:
| 192.168.008.008:01024 A 060.000.000.226:01024 E 010.000.003.254:01024 UDP
| 192.168.008.008:01024 A 060.000.000.102:01024 E 010.000.001.255:01024 UDP
| 192.168.008.008:01024 A 060.000.001.098:01024 E 010.000.002.255:01024 UDP
| 192.168.008.008:01024 A 060.000.002.234:01024 E 010.000.001.254:01024 UDP
| 192.168.008.008:01024 A 060.000.003.238:01024 E 010.000.002.254:01024 UDP
| 192.168.008.008:01024 A 060.000.001.230:01024 E 010.000.000.255:01024 UDP
Logging blocks of ports without packaging. In this case, the log includes the NAT translation address, with used block of ports and IP address of the source. For each event a new message is created.
Settings:
| 060.000.000.179:01024-01278 EA 010.000.001.253 UDP
| 060.000.003.096:01024-01278 EA 010.000.002.253 UDP
| 060.000.000.034:01024-01278 EA 010.000.000.254 UDP
| 060.000.002.245:01024-01278 EA 010.000.003.254 UDP
| 060.000.001.249:01024-01278 EA 010.000.001.255 UDP
| 060.000.000.108:01024-01278 EA 010.000.002.255 UDP
| 060.000.001.104:01024-01278 EA 010.000.000.255 UDP
| 060.000.000.253:01024-01278 EA 010.000.003.255 UDP
Logging the messages about blocks of ports release and the translation release. In this case, the last message in the example notifies of the releasing of the port 1.
Settings:
| 207.046.113.078:05443 F 060.000.003.112:01043 E 010.000.002.015:02542 TCP
| 172.016.255.001:00001 F 060.000.003.176:00001 E 067.215.065.132:00001 ICM
| 077.001.001.254:00000 A 000.000.000.000:00000 E 077.001.001.002:00001 047
Logging in hexadecimal format.
Settings:
| c0a800c10015 06 3c0002e80400 EA c0a800720471
| c0a800c11c56 06 3c0002e80401 EA c0a800720474
NetFlow logging
EcoNAT allows you to configure connection logging with NetFlow v9 protocol. In this case is logged the connection but not the amount of transferred traffic. Additional settings of connection_log branch used for this are described in the table below.
Parametr | Description |
---|---|
netflow_template_rate | It indicates after how many packets will be transmitted the netflow template package. Possible values: once, 128, 512, 1K, 4K, 16K, 64K |
netflow_options_rate | It indicates after how many packets will be transmitted netflow options and netflow options template package. Possible values: once, 128, 512, 1K, 4K, 16K, 64K |
Required to configure NetFlow logging parameter values are shown in the table below. It is recommended to strictly adhere to the specified settings.
Parametr | Value |
---|---|
log_format | netflow |
log_on_release | on |
log_individual_conn | on |
use_hex_format | off |
pack_msgs | on |
log_server | NetFlow server address and the right port number |
ip_address gateway | Address/mask of the subnet and gateway |
System logging setup
EcoNAT keeps the recordings of all user actions in the terminal console. Logs of these actions are sent to the server through the management interface.
System logging settings can be found in a system system_log branch. To turn logging on set the parameter to enable. The server on which EcoNAT will send system logs, specified in log_servers parameter.
EcoNAT name that appears in the logs is set in the hostname parameter using the command hostname "name". This name is added not only in the system log, but also in EcoNAT connection log.
MyEcoNAT:18:system.system_log# verbose defrag 1
MyEcoNAT:19:system.system_log# show
enable
log_servers ( )
hostname "econat"
timeskew 180
verbose
{
all 3
basic_nat 3
conn_track 3
defrag 1
dpi 3
fast_path 3
gc 3
health_check 3
main 3
session 3
reconfig 3
services 3
sniffer 3
snmp 3
syslogger 3
trans_tbl 3
alg 3
bras_tbl 3
}
MyEcoNAT:20:# show verboselvl
ALL = 3
BASIC_NAT = 1
CONN_TRACK = 1
DEFRAG = 1
DPI = 1
FAST_PATH = 1
GC = 1
HEALTH_CHECK = 1
MAIN = 1
RECONFIG = 1
SERVICE = 1
SNIFFER = 1
SNMP = 1
SYSLOGGER = 1
TRANS_TBL = 1
SESSION = 1
ALG = 1
BRAS_TBL = 1
In order to quickly change the logging level for some subsystem (or all subsystems), use the command setlog <subsystem> <logging rate>. Here logging levels are set no figures, as in the configuration change, but names. The changes take effect immediately. After the reboot, the logging levels values will be reset to the specified in the active configuration.
In the example below, the logging level for all subsystems is changed to FATAL, respectively, lower priority events (WARNING, INFO, ERROR) will not benefit from logging. In the configuration level of logging for all subsystems is INFO, and to after reboot will again log all events.
MyEcoNAT:21:system.system_log.verbose# setlog all fatal
MyEcoNAT:22:system.system_log.verbose# show verboselvl
ALL = 0
BASIC_NAT = 1
CONN_TRACK = 1
DEFRAG = 1
DPI = 1
FAST_PATH = 1
GC = 1
HEALTH_CHECK = 1
MAIN = 1
RECONFIG = 1
SERVICE = 1
SNIFFER = 1
SNMP = 1
SYSLOGGER = 1
TRANS_TBL = 1
SESSION = 3
ALG = 1
BRAS_TBL = 1
MyEcoNAT:23:system.system_log.verbose# ls
all 3
basic_nat 1
conn_track 1
defrag 1
dpi 1
fast_path 1
gc 1
health_check 1
main 1
session 3
reconfig 1
services 1
sniffer 1
snmp 1
syslogger 1
trans_tbl 1
alg 1
bras_tbl 1
MyEcoNAT:24:> show logs info | more
Mar 09 09:27:25 MAIN [FATAL]: User admin logged with 3
Mar 09 09:27:12 DPI [INFO]: Performed checks for short list https: total 0.00/s, allowed 0.00/s, banned 0.00/s
Mar 09 09:27:12 DPI [INFO]: buffers (min-max): state 7f3eada42980-7f3eada42980, host 0-0, path 0-0
Mar 09 09:27:12 DPI [INFO]: buffers (alloced/freed): state 1/1, host 0/0, path 0/0
Mar 09 09:27:03 GC [INFO]: abonents_table_GC_CORE_2 calls: 0, ticks: 0, ticks/entry: -nan, processed: 0, freed 0
Press any key
MyEcoNAT:25:> show logs facility snmp
May 11 12:32:50 SNMP [INFO]: Launched snmp agent on port 161 for community public
MyEcoNAT: 19: system.protocol_log # show
disable
log_interface default
server_ip_and_port 0.0.0.0
ip_address 0.0.0.0/0.0.0.0
gateway 0.0.0.0
source_port 1089
Parameters of the protocol logging are given in the table below.
Parameter | Description |
---|---|
enable disable | Enabling / disabling protocol logging |
log_interface | The interface through which logging will be carried out. Possible values: default - through the logging interface, mng - via mng interface |
server_ip_and_port | The syslog server IP address and port |
ip_address | Source IP-address and subnet mask (use '/') of the virtual channel in which logging network interfaces are merged |
gateway | The default gateway for a virtual channel, into which the logging network interfaces are merged. This setting is required if the syslog server specified in the server_ip_and_port parameter is not on the subnet specified in the ip_address parameter |
source_port | The port used for sending syslog packets |
Quality of Experience
Quality of Experience (QoE) is an integral parameter representing the general acceptability of quality or service subjectively perceived by the end user. In the context of EcoNAT, QoE is a summary of information about subscriber connections. In this summary, the indicators characterizing the quality of this connection are presented. These indicators help identify connection problems for each individual subscriber, which can be used by the operator as a tool to increase the subjective quality of the services provided and to retain subscribers.
EcoNAT QoE is divided into the following modules, which can be included both together and separately, depending on the license:
- basic functionality with binary logs;
- session accounting functionality (the number of bytes/packets transmitted is logged);
- OTT functionality, which allows analyzing the parameters for providing video services: counting bytes of the OTT sub-session, time of the last PSH packet in the sub-session from the server, delta time between the GET packet from the client and PSH from the server in the sub-session.
The QoE settings are located in the branch of the configuration tree system.qoe_log.
The QoE settings are described in the table below .
Parameter | Description |
---|---|
enable disable | Enabling / disabling QoE Logging |
log_interface | The interface through which logging will be carried out. Possible values: default - through the logging interface, mng - via mng interface |
syn_log | Possible values: on, off If the value is "on", then the passing SYN packet (including Ethernet header) will be encapsulated into a log packet with fixed DATA field length of 256 bytes, which is then forwarded to the log collector |
server_ip_and_port | <IP Address>:<Port> of Log Collector |
ip_address | Source IP-address and subnet mask (use '/') of the virtual channel in which logging network interfaces are merged |
gateway | The default gateway for a virtual channel, into which the logging network interfaces are merged. This setting is required if the syslog server specified in the server_ip_and_port parameter is not on the subnet specified in the ip_address parameter |
source_port | The port used for sending syslog packets |
mtu | MTU of syslog packets |
Settings example:
2:7:system.qoe_log# ls
enable
log_interface default
syn_log on
server_ip_and_port 192.168.1.2:514
ip_address 192.168.1.1/255.255.255.0
gateway 192.168.1.1
source_port 1089
mtu 1500
QoE logs are transmitted in binary form using a proprietary protocol. When using equipment in conjunction with EcoQoE (Log Collector), the logs are automatically decrypted at the collector.
Logging subscribers requests to web servers
The EcoNAT system provides the capability to use a remote syslog server to log HTTP GET requests, web servers HTTP responses and SSL/TLS connection requests.This functionality is configured in the system clickstream configuration branch. The table below describes the parameters available in this branch.
Parameter | Description |
---|---|
enable | Enabling / disabling logging of requests to web servers |
server_ip_and_port | The syslog server IP address and port |
ip_address | Source IP-address and subnet mask (use '/') of the virtual channel in which logging network interfaces are merged |
gateway | The default gateway for a virtual channel, into which the logging network interfaces are merged. This setting is required if the syslog server specified in the server_ip_and_port parameter is not on the subnet specified in the ip_address parameter |
source_port | The port used for sending syslog packets |
mtu | MTU of syslog packets |
Settings example:
EcoNAT:43:system.clickstream# ls
enable
server_ip_and_port 192.168.2.2:514
ip_address 192.168.1.1/255.255.255.0
gateway 192.168.1.254
source_port 1088
mtu 1500
Below is an example of records on the syslog server. The 1st record is for HTTP GET request, the 2nd is for web server HTTP response, and the 3rd is for SSL connection request.
2019-07-11T10:35:58.202901+00:00 192.168.1.1 192.168.000.002:34904 192.168.000.003:00080 1522071357 econat GET / HTTP/1.1#015#012Host: google.ru#015#012User-Agent: curl/7.55.0#015#012Accept: */*#015#012#015
2019-07-12T09:33:02.370234+00:00 192.168.1.1 065.208.228.223:00080 145.254.160.237:03372 1562934780 econat HTTP/1.1 200 OK
2019-07-15T14:50:01.810583+00:00 192.168.1.1 192.168.000.002:41016 192.168.000.003:00080 1532627400 econat SSL: 3.3 hostname: vk.com
The table below describes the values in the fields of the record for HTTP GET request (see the 1st string in the example above).
# | Field | Example |
---|---|---|
1 | Syslog server timestamp | 2018-03-26T10:35:58.202901+00:00 |
2 | EcoNAT device IP address | 192.168.1.1 |
3 | Source IP-address:port | 192.168.000.002:34904 |
4 | Destination IP address:port | 192.168.000.003:00080 |
5 | EcoNAT device timestamp (POSIX time) | 1522071357 |
6 | Hostname specified in the system_log branch | econat |
7 | HTTP GET request content | GET / HTTP/1.1#015#012Host: google.ru#015#012User-Agent: curl/7.55.0#015#012Accept: */*#015#012#015 |
The description of fields 1-6 in the record for web server HTTP response (see the 2nd string in the example above) is the same as for HTTP GET request. The field 7 contains HTTP version and response status code.
The table below describes the fields 7 and 8 of the record for SSL connection request (see the 3rd string in the example above). The description of fields 1-6 is the same as for HTTP GET request.
# | Field | Example |
---|---|---|
7 | SSL version | SSL: 3.3 |
8 | Domain name | hostname: vk.com |
To view the statistics on packets for logging of requests to web servers, use the command show counters all | include clickstream. The counters displayed by this command are described in the table below.
Counter | Description |
---|---|
cr_clickstream_url_for_log | Prepared syslog packets |
cr_clickstream_send_one_packet | Sent syslog packets |
cr_clickstream_send_fragmented_packet | Sent fragmented syslog packets |
cr_clickstream_error_general | The number of errors occured when cloning a TCP packet |
cr_clickstream_error_create_header | The number of errors occured when creating a syslog packet |
cr_clickstream_warn_invalid_sequence | The number of received TCP packets with invalid sequence number |
cr_clickstream_error_no_session | The number of received TCP packets for which a record in the session table was not found |
cr_clickstream_no_ssl_tmp_buffer | The size of buffer dedicated for ClientHello |
cr_clickstream_ssl_without_hostname | The number of received SSL or TLS handshakes without hostname |
Example:
EcoNAT:10:> show counters all | include clickstream
Core total, cr_clickstream_url_for_log: 11
Core total, cr_clickstream_send_one_packet: 11
Core total, cr_clickstream_error_no_session: 11