[eberger]

Insider 3.1 Build 5372: With this version of insider we get very often the message "... has been stuck processing request". The monitoring targets are HP-UX servers/databases. What does this message exactly mean? What can be the cause for this message?

Example message:
[07/30/10 08:12:04] ora-sapp50.lksdom21.lks.local
(ora-sapp50.lksdom21.lks.local@orainfo:22) has been stuck processing request
memory-info() [07:30:2010 08:08:44:533]. Disconnected.
_________________
Kind regards,
Ernst Berger

[skavish]

It means that this request has been timed out. The timeout is specified in settings.xml in 'requestServerTimeoutInMinutes'.

In this particular case I believe the problem is not in the timeout itself (it seems it was hanging for about 4 minutes: from 8:08:44 to 8:12:04), but rather in the request itself. This particular request on HP-UX is issuing the following commands:

vmstat 1 2

then it may execute:

grep -i physical: /var/adm/syslog/syslog.log

it may also execute:

/usr/sbin/dmesg | grep -i physical:

and finally:

sudo /usr/sbin/dmesg | grep -i physical:

Note that normally not all those commands are executed.

Please try to execute them under the same user account and check if you see any problems. If any of those commands asks for a input (like sudo may) then it will be stuck when executed from within Insider.

Please let me know if this is the case. We are working on making this stuff more configurable.

Thank you!
_________________
Dmitry Skavish
FourthElephant, Inc.

[skavish]

Ernst, could you please submit logs to us? You can do it from Help->Submit Bug. Thank you!
_________________
Dmitry Skavish
FourthElephant, Inc.

[eberger]

I've done some research, it must be an issue of our HP-UX boxes. In most of the HP-UX boxes the memory information is present in /var/adm/syslog/syslog.log, but in some of them the memory information is missing, i don't kow why.

On those where the memory information is missing the command "sudo /usr/sbin/dmesg ..." does not work because the user is not in sudoers file, if i put the user into sudoers it works - but we do not want to do this because in our configuration the admins get an email for every sudo-command.

I will open a call at HP to find out why the memory information is missing sometimes.
_________________
Kind regards,
Ernst Berger

[skavish]

Ernst, it would be great if you could share HP's response here. Thank you!
_________________
Dmitry Skavish
FourthElephant, Inc.

[eberger]

In HP-UX for dmesg the buffer size is about 8k, this is to less in our configuration. Therefore it is no memory information in dmesg and in syslog.log. Futher information can be found in HP case no. 4617879021.

The following setting in /sbin/init.d/syslogd should put memory information into syslog.log after the next reboot:

[code]
/usr/sbin/syslogd $SYSLOGD_OPTS &&
echo "System message logger started"
/sbin/dmesg | /usr/bin/logger -t DMESG --> add this line!
set_return
[/code]

In the meantime i took the memory info from swapinfo and wrote it into syslog via the logger-command:

[code]
logger "physical page size = 4096 bytes, logical page size = 4096 bytes"
logger "Physical: 37614516 Kbytes, lockable: 15664204 Kbytes, available: 21950312 Kbytes"
[/code]

... this works fine.
_________________
Kind regards,
Ernst Berger

[skavish]

Thanks for the information!
_________________
Dmitry Skavish
FourthElephant, Inc.