Admin: Graphs

From Resin 3.0

Jump to: navigation, search


Contents

Graphs

The Graphs tab in the /resin-admin gives you a view of the meter data collected by Resin across the cluster. The Statistics service that gathers the Meter data is available in Resin Professional.

Graph Browsing

When looking at the server statistics, you first need to select meters to display. Resin's meters are the statistics data streams gathered every minute and stored by the triad servers. Some meters are JMX attributes collected over time, and others record data from Resin's embedded sensors.

Selecting a meter will add the meter to the graph. Once you've selected a group of meters, you can save them as a named meter set using meter save.

Meter Names

Meter names follow a standard convention: "00|Author|Group|Name". The "00" is the server index in the cluster. Author is soemthing like "JVM", "OS", or "Resin", or "MyCom" for custom meters.

Server Groups

Graphs for servers in the cluster can be displayed in three basic modes: single server, one graph, or multiple graphs. In the single graph mode, each meter for each server has its own graph line. In the multiple graph mode, each server gets its own graph. The multiple-graph mode is more useful comparisons across the cluster.

Cookbook: setting up a thread graph

  1. Clear all the meters by clicking the "Clear Meters" button on the right.
  2. Open the "JVM|Thread" group to find the recorded data from the JVM's own thread count.
  3. Select "JVM Thread Count". You should see a graph of the JVM's thread count in the graph. You can use the "Time" selector to change the timescale to use.
  4. Open the "Resin|Thread" group for the meters in Resin's own thread pool.
  5. Select all the meters in the "Resin|Thread" group. You should see a graph with about 4 lines visible and the rest at zero.
  6. Type "threads" in the Meter Save Name form and select "Save Meters". Saving the meters will add "threads" as a predefined meter group in the Meters selection at the top.

Graphs-threads.png

Meters

The predefined meters are in three groups: JVM, OS, and Resin.

  • JVM is data from the JVM's JMX beans, like thread counts and garbage collection.
  • OS is data from the operating system, like CPU counts.
  • Resin is data from Resin's JMX and sensors

JVM|Compilation

The JVM compilation group measure JIT compilation times as reported by the JVM.

Compilation Time
the time taken for garbage collection in the last 60 seconds.

JVM|Memory

The JVM's memory and garbage collection information is useful when tuning memory and checking for memory leak situations, and checking that GC time is in a reasonable range.

GC Time|PS MarkSweep
the GC time taken in the last 60 seconds for full mark-sweep collection as reported by the JVM.
GC Time|PS Scavenge
the GC time taken for short GC scavenging as reported by the JVM.
Heap Memory Free
free heap memory in bytes as reported by the JVM
Heap Memory Used
total allocated memory in the heap
Loaded Classes
total number of classes loaded by the JVM
PermGen Memory Free
memory free in the "perm gen" group, used for .class data
PermGen Memory Used
allocated memory in the perm gen pool.
Tenured Memory Free
free memory in the long-term tenured heap
Tenured Memory Used
allocated memory in the long-term tenured heap

JVM|Thread

The JVM's thread group reports the total number of threads in the JVM.

JVM Thread Count
The total threads in the JVM

OS|CPU

The CPU load as reported by the JVM. This report is different for different operating systems. On Linux, the CPU is reported for each CPU and combined.

Unix Load Avg
On Unix systems (non-Linux), reports the system's Load Average. The load average is the count of runnable processes; it's not directly a CPU load measure

OS|Memory

The OS|Memory group report operating system memory.

Physical Memory Free
The physical free memory as reported in JMX.
Swap Free
The free swap as reported in JMX.

OS|Process

Process-related information as reported by the OS.

File Descriptor Count
The number of open files and sockets in the JVM process

Resin|Cache

The cache statistics include both the proxy cache and Resin's underlying block cache, which is also used for distributed sessions.

Block Miss Count
How many times the low-level block cache missed, causing Resin to read or write from disk
Block Read Count
The count of blocks read in the last 60 seconds.
Block Write Count
How many blocks were written to disk in the last 60 seconds.
Proxy Cache Hit Count
How many requests successfully used the proxy cache in the last 60 seconds.
Proxy Cache Miss Count
How many cacheable requests failed to find a valid page in the proxy cache in the last 60 seconds.

Troubleshooting

A high block read or write count may indicate that the block cache is too small. Since the purpose of the block cache is to reduce the slow filesystem reads and writes, high block reads and writes means Resin is spending more time reading and writing files.

Resin|Cluster

The Cluster group measures outgoing connections to other servers in the Resin system. This measurement is similar to the heartbeat since it counts cluster connections.

There are separate meters for each outgoing server. So server #2 will have data going to servers #0 and #1.

Connection Active|NN cluster-id
Measure the current active connections from this server to a target server named by "NN:cluster-id"
Connection Count|NN cluster-id
Counts the number of connections created in the last 60s from this server to the target server.
Idle Active|NN cluster-id
Counts the current idle connections in the pool from this server to the target server.
Idle Count|NN cluster-id
Counts the number of transitions to the idle state.
Request Active|NN cluster-id
The current number of active requests from this server to the target server
Request Count|NN cluster-id
The number of requests to the target server in the last 60s.
Request Fail|NN cluster-id
The number of failed requests to the target server in the last 60s
Request Time|NN cluster-id
The average request time for requests to the target server in the last 60s.
Request Time Max
NN cluster-id
The longest request time for a request to the target server in the last 60s
Request Time 95%
NN cluster-id
The time for 95% of requests to complete

Resin|Database

The data for the Resin database pool lets you tune the pool, and check for slow database queries.

Query Active
Counts the current number of active queries.
Query Count
Counts the queries in the last 60s
Query Time
The average query time in the last 60s
Query Time Max
The maximum query time in the last 60s
Query Time 95%
The time for 95% of queries to complete
Connection Active
The current number of active database connections
Connection Count
The number of connections created in the last 60s
Connection Time
The average open time for connections in the last 60s.
Idle Active
The current number of idle connections in the database pool
Idle Count
The number of connections changing to the idle state in the last 60s
Idle Time
The average time connections are idle for the last 60s.

Resin|Health

Each health check in Resin's health system records its current status as a meter. Since the "OK" level is zero, a stable system has a zero graph. The warning level is 1 and the fail level is 2.

Resin|Http

HTTP requests and sessions are recorded in the Resin|Http section, letting you check for slow requests and unexpected HTTP session sizes.

Request Active
The current number of active requests
Request Bytes
The number of bytes transferred in the last 60s
Request Count
The number of requests in the last 60s
Request Time
The average time for a request in the last 60s
Request Time Max
The slowest request in the last 60s
Request Time 95%
The time for 95% of requests to complete
Session Save Count
The number of sessions saved in the last 60s
Session Save Size
The average serialized session size in the last 60s

Resin|Thread

Statistics related to the Resin thread pool, used for requests and timers.

Thread Active Count
The number of threads currently active
Thread Count
The total number of threads managed by Resin
Thread Create Count
The threads created by Resin in the last 60s
Thread Idle Count
The current number of threads idle in the pool.
Thread Overflow Count
The number of threads Resin created using the overflow method.
Thread Priority Queue
The number of threads dispatching the priority queue
Thread Starting Count
The current number of threads starting
Thread Task Queue
The number of threads reading from the task queue
Thread Wait Count
The requests waiting for an active thread

Troubleshooting

  • The thread create count should generally be very low, and preferably zero. If the create count is high, the pool isn't being effective.
  • The overflow count should be zero unless the pool is overflowing.
  • The Priority Queue and Task Queue counts should generally be zero unless there's a thread spike.
Personal tools