http://wiki3.caucho.com/index.php?title=Watchdog&feed=atom&action=historyWatchdog - Revision history2024-03-28T21:32:21ZRevision history for this page on the wikiMediaWiki 1.18.0http://wiki3.caucho.com/index.php?title=Watchdog&diff=3513&oldid=prevReza at 18:04, 17 December 20092009-12-17T18:04:15Z<p></p>
<a href="http://wiki3.caucho.com/index.php?title=Watchdog&diff=3513&oldid=3490">Show changes</a>Rezahttp://wiki3.caucho.com/index.php?title=Watchdog&diff=3490&oldid=prevReza: New page: <document> <header> <product>resin</product> <title>Resin Watchdog</title> <type>contents</type> <description> <p> For reliability and security, Resin servers are starte...2009-12-17T08:46:21Z<p>New page: <document> <header> <product>resin</product> <title>Resin Watchdog</title> <type>contents</type> <description> <p> For reliability and security, Resin servers are starte...</p>
<p><b>New page</b></p><div><document><br />
<header><br />
<product>resin</product><br />
<title>Resin Watchdog</title><br />
<type>contents</type><br />
<br />
<description><br />
<p><br />
For reliability and security, Resin servers are started and monitored<br />
by a separate Resin watchdog process. The watchdog continually checks<br />
the health of<br />
the Resin server and restarts the Resin instance if is becomes unresponsive.<br />
</p><br />
<br />
<p>In most cases, the watchdog reads the resin.xml and configures itself<br />
automatically, so no extra configuration is required. Certain specialized<br />
configurations like ISPs can configure the watchdog to isolate JVMs<br />
for protection and security.<br />
</p><br />
</description><br />
</header><br />
<br />
<body><br />
<br />
<localtoc/><br />
<br />
<s1 title="Overview"><br />
<br />
<p>Because the watchdog runs quietly as a separate service, most of<br />
the time you won't need to pay attention to the watchdog at all. The<br />
standard configuration launches on watchdog per machine which<br />
monitors all the Resin JVMs on that matchine, so most sites will not<br />
need to change any watchdog configuration.</p><br />
<br />
<p>The main management tasks where you might need to pay attention to<br />
the watchdog is shutting it down if something is severely wrong with<br />
the machine, and checking the watchdog logs for Resin restart<br />
events.</p><br />
<br />
<p>The watchdog automatically restarts Resin if the Resin JVM ever<br />
crashes or exits. So if you want to stop Resin, you need to tell the<br />
watchdog to stop the instance, or you can stop the watchdog entirely.<br />
The watchdog is typically controlled by the resin.jar main<br />
program (<code>ResinBoot</code>), which has commands to start, stop,<br />
and restart Resin instances as well as reporting the watchdog<br />
status.</p><br />
<br />
<figure src="startup-watchdog.png"/><br />
<br />
<p>While most users will use the watchdog automatically with no extra<br />
configuration, ISPs and larger and complicated sites can create a<br />
specialised watchdog.xml with a &lt;watchdog-manager> tag to control<br />
the watchdog at a much finer level. The &lt;watchdog-manager> lets an<br />
ISP run the watchdog under its own control, and specify exactly the<br />
command-line parameters for their users' Resin instances, including<br />
the ability to create secure chroot instances for their users.<br />
Typically, the watchdog will run as root, while the user instances<br />
will run with their respective user ids.</p><br />
<br />
</s1><br />
<br />
<s1 title="command-line"><br />
<br />
<p>The watchdog is controlled on the command-line using resin.jar's<br />
main class, <code>ResinBoot</code>. The major operations are: start, stop, restart, shutdown<br />
and status.</p><br />
<br />
<s2 title="console"><br />
<br />
<p>The "console" command starts a new Resin instance in a console window for<br />
development. The standard output of the Resin instance will appear in the<br />
console window.</p><br />
<br />
<example title="Example: watchdog console"><br />
resin-4.0.x> java -jar lib/resin.jar -conf conf/test.conf -server a console<br />
...<br />
</example><br />
<br />
</s2><br />
<br />
<s2 title="start"><br />
<br />
<p>The "start" command starts a new Resin instance with the given server id.<br />
<code>ResinBoot</code> will first try to contact the watchdog on the<br />
current machine, and start a new watchdog if necessary. The server id<br />
must be unique for all servers defined in the resin.xml.</p><br />
<br />
<example title="Example: watchdog start"><br />
resin-4.0.x> java -jar lib/resin.jar -conf conf/test.conf -server a start<br />
<br />
Resin/4.0.x started -server 'a' for watchdog at 127.0.0.1:6700<br />
</example><br />
<br />
</s2><br />
<br />
<s2 title="stop"><br />
<br />
<p>The "stop" command stops the Resin instance with the given server id.<br />
If the stopped instances is the last one managed by the watchdog, the<br />
watchdog will automatically exit. If no <code>-server</code> is specified,<br />
the watchdog defaults to <code>-server ""</code>.</p><br />
<br />
<example title="Example: watchdog stop"><br />
resin-4.0.x> java -jar lib/resin.jar stop<br />
<br />
Resin/4.0.x started -server '' for watchdog at 127.0.0.1:6600<br />
</example><br />
<br />
</s2><br />
<br />
<s2 title="status"><br />
<br />
<p>The "status" command summarizes the current Resin instances managed<br />
by the watchdog service.</p><br />
<br />
<example title="Example: watchdog status"><br />
resin-4.0.x> java -jar lib/resin.jar status<br />
<br />
Resin/4.0.x status for watchdog at 127.0.0.1:6600<br />
<br />
server '' : active<br />
password: missing<br />
user: ferg<br />
root: /home/test/resin/<br />
conf: /etc/resin/resin.xml<br />
</example><br />
<br />
</s2><br />
<br />
</s1><br />
<br />
<s1 title="Single Resin instance"><br />
<br />
<p>This example shows a single-server site listening to the<br />
standard HTTP port 80 and running the server as the "resin" user. In this<br />
example, the watchdog typically runs as root so it can bind to the<br />
protected port 80, while the Resin instance runs as "resin" for security.</p><br />
<br />
<figure src="watchdog-single.png"/><br />
<br />
<p>Since this configuration uses the default, the watchdog listens to<br />
port 6600 for commands.</p><br />
<br />
<example title="Example: /etc/resin/resin.xml"><br />
&lt;resin xmlns="http://caucho.com/ns/resin"<br />
xmlns:resin="urn:java:com.caucho.resin"><br />
<br />
&lt;cluster id=""><br />
<br />
&lt;server id="app-a" address="127.0.0.1"><br />
&lt;user-name>resin&lt;/user-name><br />
&lt;group-name>resin&lt;/group-name><br />
<br />
&lt;http port="80"/><br />
&lt;/server><br />
<br />
&lt;resin:import path="${__DIR__}/app-default.xml"/><br />
<br />
&lt;host id=""><br />
&lt;web-app id="" path="/var/www/htdocs"/><br />
&lt;/host><br />
<br />
&lt;/cluster><br />
&lt;/resin><br />
</example><br />
<br />
</s1><br />
<br />
<s1 title="Single machine load balance with shared watchdog"><br />
<br />
<p>When running multiple instances of Resin on the same<br />
machine, one watchdog-manager typically handles all the instances.<br />
The server id will select which instance to start or stop.</p><br />
<br />
<p>In this example, there is one web-tier server acting as a load-balancer<br />
and two app-tier servers handling the backend data, all on a single machine.<br />
A site might want multiple app-tier servers for more reliable maintenance<br />
and upgrades. While one server is down, traffic can be handled by a second<br />
server.</p><br />
<br />
<p>The example uses default watchdog configuration from the standard resin.xml<br />
file. The watchdog process and <code>ResinBoot</code> will both read<br />
the resin.xml file for the server configuration, so there's no explicit<br />
watchdog configuration necessary. The watchdog detects that<br />
multiple servers are running on the same machine and manages all of them<br />
automatically.</p><br />
<br />
<br />
<figure src="watchdog-multiple.png"/><br />
<br />
<example title="Example: /etc/resin/resin.xml"><br />
&lt;resin xmlns="http://caucho.com/ns/resin"<br />
xmlns:resin="urn:java:com.caucho.resin"><br />
<br />
&lt;cluster id="app-tier"><br />
<br />
&lt;server-default><br />
&lt;user-name>resin&lt;/user-name><br />
&lt;group-name>resin&lt;/group-name><br />
&lt;/server-default><br />
<br />
&lt;server id="app-a" address="192.168.1.10" port="6810"/><br />
&lt;server id="app-b" address="192.168.1.10" port="6811"/><br />
<br />
&lt;host id=""><br />
&lt;web-app id="" path="/var/www/htdocs"/><br />
&lt;/host><br />
<br />
&lt;/cluster><br />
<br />
&lt;cluster id="web-tier"><br />
<br />
&lt;server-default><br />
&lt;user-name>resin&lt;/user-name><br />
&lt;group-name>resin&lt;/group-name><br />
&lt;/server-default><br />
<br />
&lt;server id="web-a" address="192.168.1.10" port="6800"><br />
&lt;http port="80"/><br />
&lt;/server><br />
<br />
&lt;host id=""><br />
<br />
&lt;resin:LoadBalance regexp="" cluster="app-tier"/><br />
<br />
&lt;/host><br />
<br />
&lt;/cluster><br />
<br />
&lt;/resin><br />
</example><br />
<br />
</s1><br />
<br />
<s1 title="Single machine load balance with distinct watchdog"><br />
<br />
<p>In some cases, it's best to let each Resin instance have its own<br />
watchdog, for example when multiple users are sharing the same<br />
machine. Each &lt;server> block configures a separate &lt;watchdog-port>.<br />
Because the watchdog will read the resin.xml and use the &lt;server> block<br />
matching the <code>-server id</code> command-line argument, each watchdog<br />
will start with it's own port.</p><br />
<br />
<example title="Example: /etc/resin/resin.xml"><br />
&lt;resin xmlns="http://caucho.com/ns/resin"><br />
<br />
&lt;cluster id="app-tier"><br />
<br />
&lt;server-default><br />
&lt;user-name>resin&lt;/user-name><br />
&lt;group-name>resin&lt;/group-name><br />
&lt;/server-default><br />
<br />
&lt;server id="app-a" address="192.168.1.10" port="6810"><br />
&lt;watchdog-port>6700&lt;/watchdog-port><br />
<br />
&lt;http port="8080"/><br />
&lt;/server><br />
<br />
&lt;server id="app-b" address="192.168.1.10" port="6811"><br />
&lt;watchdog-port>6701&lt;/watchdog-port><br />
<br />
&lt;http port="8081"/><br />
&lt;/server><br />
<br />
&lt;host id=""><br />
&lt;web-app id="" path="/var/www/htdocs"/><br />
&lt;/host><br />
<br />
&lt;/cluster><br />
<br />
&lt;/resin><br />
</example><br />
<br />
<p>In the previous example, starting Resin with <code>-server app-a</code><br />
will start a watchdog at port 6700. Starting Resin<br />
with <code>-server app-b</code> will start the watchdog at port 6701.</p><br />
<br />
<example title="Example: starting app-b with watchdog-port=6701"><br />
resin-4.0.x> java -jar lib/resin.jar -server app-b start<br />
</example><br />
<br />
</s1><br />
<br />
<s1 title="ISP watchdog management"><br />
<br />
<p>In a situation like an ISP, you may wish to have a separate<br />
configuration file for the watchdog, which launches Resin instances<br />
for different users. In this case, you will want to make sure<br />
the watchdog.xml is not readable by the users, and make sure to<br />
set a management user (see <a href="security.xtp">resin-security</a>).</p><br />
<br />
<ul><br />
<li>Start and restart the user's Resin JVM</li><br />
<li>Set JVM parameters and Java executable</li><br />
<li>Set the Resin instance root-directory</li><br />
<li>setuid user-name and group-name</li><br />
<li>Set the resin.xml configuration (must be readable by the user)</li><br />
<li>Open protected ports like port 80</li><br />
<li>Optional chroot for additional security</li><br />
</ul><br />
<br />
<p>The watchdog will launch the Resin instance with the given user as a<br />
setuid. It will also open any necessary protected ports, e.g. port 80.</p><br />
<br />
<example title="Example: /etc/resin/watchdog.xml"><br />
&lt;resin xmlns="http://caucho.com/ns/resin"<br />
xmlns:resin="urn:java:com.caucho.resin"><br />
<br />
&lt;resin:AdminAuthenticator><br />
&lt;user name="harry" password="MD5HASH=="/><br />
&lt;/resin:AdminAuthenticator><br />
<br />
&lt;watchdog-manager><br />
<br />
&lt;watchdog-default><br />
&lt;jvm-arg>-Xmx256m&lt;/jvm-arg><br />
&lt;/watchdog-default><br />
<br />
&lt;watchdog id="user_1"><br />
&lt;user-name>user_1&lt;/user-name><br />
&lt;group-name>group_1&lt;/group-name><br />
<br />
&lt;resin-xml>/home/user_1/conf/resin.xml&lt;/resin-conf><br />
&lt;resin-root>/home/user_1/www&lt;/resin-root><br />
<br />
&lt;open-port address="192.168.1.10" port="80"/><br />
&lt;/watchdog><br />
<br />
...<br />
<br />
&lt;watchdog id="user_n"><br />
&lt;user-name>user_n&lt;/user-name><br />
&lt;group-name>group_n&lt;/group-name><br />
<br />
&lt;resin-conf>/home/user_n/conf/resin.xml&lt;/resin-conf><br />
&lt;resin-root>/home/user_n/www&lt;/resin-root><br />
<br />
&lt;open-port address="192.168.1.240" port="80"/><br />
&lt;/watchdog><br />
<br />
&lt;/watchdog-manager><br />
<br />
&lt;/resin><br />
</example><br />
<br />
</s1><br />
<br />
<s1 title="Management/JMX"><br />
<br />
<p>The watchdog publishes the watchdog instances to JMX with the JMX name<br />
"resin:type=Watchdog,name=a". With a JMX monitoring tool like jconsole,<br />
you can view and manage the watchdog instances.</p><br />
<br />
</s1><br />
<br />
</body><br />
</document></div>Reza