Hvrmaint

From HVR
Jump to: navigation, search
Commands
Previous: hvrlogrelease
Next: hvrproxy
Command Reference

Name

hvrmaint – Housekeeping script for HVR on the hub machine.

Synopsis

hvrmaint [optfile] [–options]

Description

Command hvrmaint is a script for regular housekeeping of the HVR on the hub machine. The behavior of hvrmaint is controlled by its options. These can be supplied on its command line, or they can be bundled into a file optfile, which must be its first argument.

One way to use hvrmaint is to schedule it on its own with options –stop and –start, perhaps nightly or each weekend. These options instruct hvrmaint to restart the HVR Scheduler. Often other options would be supplied such as –scan_hvr_out, (scan log files for HVR errors) or –archive_dir (move old log files to archive directory). Option –email can be used to send an email to operator(s) that summarizes the status. When used in this way hvrmaint could be scheduled on Unix using crontab, and on Windows as a Windows Scheduled Task.

A second way to use hvrmaint is to run it frequently (perhaps every half hour) with options –ping, –check_logfile_growth and –scan_hvr_out. These options check that the HVR Scheduler is responding, that its logfile is growing and that no errors have been written. Running hvrmaint this way does not interrupt the HVR Scheduler. Option –email_only_when_errors can be supplied so that emails are only sent if a problem is found.

The last way to use hvrmaint is as part of a larger nightly or weekly batch script, perhaps a script which halts all server processes (including the DBMS), does a system backup and then restarts everything again. In this case hvrmaint would be called at the top of the batch script with option –stop (stop the HVR Scheduler) and would then be called again near the bottom with option –start (restart the HVR Scheduler).

Note: Command hvrmaint cannot process log files containing more than 12 months of data.
SC-Hvr-Maintenance weekly.png

Options

Parameter Description
–task_name=task Task name is used internally by hvrmaint to locate its option file and name its offset files. This allows different tasks defined in the GUI to have a different state. e.g. so that a when a task for one channel has processed today's files a different task for a different channel still remembers to process today's files.
–scan_hvr_out Scan Scheduler log file hvr.out. Command hvrmaint writes a summary of HVR errors detected in this file to its output and to any emails that it sends.
–scan_channel=chn Only scan for general errors and errors in specified channel(s) chn.
–scan_location=loc Only scan for general errors and errors in specified locations(s) loc.
–scan_ignore=patt Ignore log records which match specified pattern patt (can be regular expression).
–test_scheduler Check that HVR Scheduler is actually running using hvrtestscheduler. If option -stop is also defined then this test is performed before the HVR Scheduler is stopped. If option -start is supplied then hvrmaint always checks that the HVR Scheduler is running using a test, regardless of whether or not option -test_scheduler is defined.
–check_logfile_growth Check that logfile hvr.out has grown in size since the last time hvrmaint was run. If this file has not grown than an error message will be written.
–task_group=group Task group allows different hvrmaint tasks to share the same state. So a nightly task that processes logfiles and gives a warning if the latency is >1 hour can use the same 'offset state' as a task that runs during the day which gives a warning if latency is >1 minute.
–latency_limit=dur Check for replication latencies and consider jobs over the limit erroneous. Value for dur can be specified in one of the following formats:
  • N d or N day or N days - Indicates number of days
  • N h or N hour or N hours - Indicates number of hours
  • N m or N min or N mins or N minute or N minutes - Indicates number of minutes
  • N s or N sec or N secs or N second or N seconds - Indicates number of seconds
  • HH:MM:SS[.SSS] - Indicates time format with hour, minute, second, and millisecond
  • MM:SS[.SSS] - Indicates time format with minute, second, and millisecond
Note: Milliseconds [.SSS] is ignored by this parameter.
–latency_channel=chn Only check latencies of jobs in specified channel(s) chn.
–latency_location=loc Only check latencies of jobs in specified location(s) loc.
–stop Stops HVR Scheduler.
–start Starts HVR Scheduler.
–quiesce_grace=secs If jobs are still running when the HVR Scheduler must stop, allow seconds secs grace before killing them. The default is 60 seconds. This parameter is passed the HVR Scheduler using the –q option.
–archive_files=patt Move any files in directory $HVR_CONFIG/log/hub with pattern patt to the archive directory. Files which are not matched by patt are deleted.
–archive_keep_days=N Removed files in archive directory after N number of days. Requires option –archive_files. If this option is not specified, then archived files are kept indefinitely.
–archive_compress Compress HVR Scheduler log files while moving them to the archive directory. For a Windows hub, this option can only be used if command gzip has been installed.
–journal_keep_days=n Remove HVR journal files in directory $HVR_CONFIG/jnl after n number of days. These files are written by integrate jobs if parameter Integrate /JournalRouterFiles is defined.
–output_verbose Prints latency information and error summaries to hvrmaint output.
–email_to=addr1[;addr2] Email output from hvrmaint output to addr1 [and addr2]. Requires either option –smtp_server or option –mailer. Multiple email addresses can be specified, either using a single –email_to option with values separated by a semicolon or using multiple –email_to options.
–email_from=addr Specify a sender address addr in email header.
–email_only_when_errors Only send an email if hvrmaint encountered an error itself or detected an HVR error while scanning hvr.out.
–email_only_when_errors_or_warnings Only send an email if hvrmaint encountered an error itself or detected an HVR error or warning while scanning hvr.out.
–error_limit=N HVR errors reported is limited to N number. This option prevents the generated emails becoming too large. Default is 1000.
–smtp_server=server SMTP server to use when sending email. Value server can be either a node name or IP address. Requires option –email.
–smtp_user=user Username user for authentication SMTP server if needed.
–smtp_pass=pass Password pass used for authentication on the SMTP server if needed.
–mailer=cmd Mailer command to use for sending emails, instead of sending them via an SMTP server. Requires option –email. String %s contained in cmd is replaced by the email subject and string %a is replaced by the intended recipients of the email. The body of the email is piped to cmd as stdin. E.g. on Linux: –mailer=/bin/mail -s %s %a
–slack_webhook_url=url A webhook for a Slack channel in company MyCorp looks like https://hooks.slack.com/services/xxxx/yyyy.

To generate such a webhook, sign into Slack, then navigate to Apps ▶ Manage apps ▶ Custom Integrations ▶ Incoming WebHooks ▶ Add Configuration.

–slack_channel=chn Hvrmaint will send the message to the specified slack channel chn. You can also send the message to a user (@username).
–send_slack_only_when_errors Only send a slack message if hvrmaint encountered an error itself or detected an HVR error while scanning hvr.out.
–send_slack_only_when_errors_or_warnings Only send a slack message if hvrmaint encountered an error itself or detected an HVR error or warning while scanning hvr.out.
–snmp_notify Send SNMP v1 traps or v2c notifications. The -snmp_community option is required. See $HVR_HOME/lib/mibs/HVR-MIB.txt
–snmp_version=vers Specify SNMP version. Value vers can be '1' or '2c' (default).
–snmp_heartbeat Send a hvrMaintNotifySummary notification, even if there was nothing to report.
–snmp_hostname=host SNMP agent hostname host. Defaults to localhost.
–snmp_port=port SNMP agent trap port. Defaults to port 162.
–snmp_community=str Community string str for SNMPv1/v2c transactions.
–disable hvrmaint is running but alerts are not generated. This can be useful during a maintenance window when channels are being modified or stopped. An alternative is to stop running hvrmaint during the maintenance window and restart it after, but this can generate a lot of alerts caused by the maintenance.
–hub=hub Hub database for HVR Scheduler. This value has form user/pwd (for an Oracle hub), inghub (for an Ingres hub database), or hub for a (SQL Server hub database). For Oracle, passwords can be encrypted using command hvrcrypt. In HVRGUI, this option is available inside the Text tab.
–sched_option=schedopt Extra startup parameters for the HVR Scheduler service. Possible examples are –uuser/pwd (for a username), –hsqlserver (for the hub class) or –cclus/clusgrp (for Windows cluster group). In HVRGUI, this option is available inside the Text tab.
–output=fil Append hvrmaint output to file fil. If this option is not supplied, then output is sent to stdout. Output can also be sent to an operator using option –email. In HVRGUI, this option is available inside the Text tab.
–env=NAME=VALUE Set environment variable. This option can be repeated to set multiple variables such as $HVR_HOME, $HVR_CONFIG, $HVR_TMP, $II_SYSTEM, $ORACLE_HOME etc. In HVRGUI, this option is available inside the Text tab.

Files

Folder-icon.png HVR_CONFIG
├─ Folder-icon.png log
└─ Folder-icon.png hubdb HVR Scheduler log files.
├─ hvr.out Main Scheduler log file.
└─ .hvrmaint_state hvrmaint state file.
└─ Folder-icon.png logarchive
└─ Folder-icon.png hubdb
└─ Folder-icon.png YYYYMMDD
├─ hvr.out Archived Scheduler log file. These files are created if hvrmaint option –archive_files is defined and deleted again if option –archive_keep_days is defined.
└─ hvr.out.gz Archived Scheduler log file if –archive_compress is defined.

Examples

Unix & Linux

On Unix hvrmaint could be scheduled to monitor the status of HVR every hour and also to restart the HVR Scheduler and rotate log files at 21:00 each Saturday. The environment for such batch programs is very limited, so many –env options are needed to pass it sufficient environment variables.

Two option files are prepared. The first option file /usr/hvr/hvr_config/files/hvrmaint_hourly.opt will just check for errors and contains the following:

–hub=hvr/!{s8Dhx./gsuWHUt}!			# Encrypted Oracle password 
–sched_option=-h oracle
–env=HVR_HOME=/usr/hvr/hvr_home
–env=HVR_CONFIG=/usr/hvr/hvr_config
–env=HVR_TMP=/tmp
–env=ORACLE_HOME=/distr/oracle/OraHome817
–env=ORACLE_SID=ORA817
–email_from=hvr@prod.mycorp.com
–email_to=bob@mycorp.com;jim@mycorp.com
–email_only_when_errors
–snmp_server=snmp.mycorp.com
–output=/usr/hvr/hvr_config/files/hvrmaint.log
–scan_hvr_out

The second option file /usr/hvr/hvr_config/files/hvrmaint_weekly.opt will restart the HVR Scheduler and rotate the log files each week.

–hub=hvr/!{s8Dhx./gsuWHUt}!				# Encrypted Oracle password 
–sched_option=-h oracle
–env=HVR_HOME=/usr/hvr/hvr_home
–env=HVR_CONFIG=/usr/hvr/hvr_config
–env=HVR_TMP=/tmp
–env=ORACLE_HOME=/distr/oracle/OraHome817
–env=ORACLE_SID=ORA817
–email_from=hvr@prod.mycorp.com
–email_to=bob@mycorp.com;jim@mycorp.com
–email_only_when_errors
–snmp_server=snmp.mycorp.com
–output=/usr/hvr/hvr_config/files/hvrmaint.log
–scan_hvr_out
–stop
–archive_files=hvr.out				# Only archive log file hvr.out
–archive_compress
–archive_keep_days=14		 	 	# Delete files after 2 weeks
–journal_keep_days=4
–start

The following lines are added to crontab for user hvr (these should be single lines without wrapping):

0 * * * * /usr/hvr/hvr_home/bin/hvrmaint /usr/hvr/hvr_config/files/hvrmaint_hourly.opt 
0 21 * * * /usr/hvr/hvr_home/bin/hvrmaint /usr/hvr/hvr_config/files/hvrmaint_weekly.opt

Alternatively the following line could be added to crontab for root:

0 21 * * 6 su hvr –c /usr/hvr/hvr_home/bin/hvrmaint /usr/hvr/hvr_config/files/hvrmaint_weekly.opt

Instead of scheduling hvrmaint on its own, it could also be used as part of a larger nightly batch script run by root which halts the HVR Scheduler and DBMS before doing a system backup. This batch script would roughly look like this:

su hvr –c /usr/hvr/hvr_home/bin/hvrmaint /opt/hvrmaint.opt –stop –scan_hvr_out –archive_files=hvr.out
su ingres –c /opt/ingres/utility/ingstop # Stop DBMS backup –f/dev/rmt/0m # Perform system backup
su ingres –c /opt/ingres/utility/ingstart # Restart DBMS
su hvr –c /usr/hvr/hvr_home/bin/hvrmaint /opt/hvrmaint.opt –start


Windows

On Windows, hvrmaint can be run as a Windows Scheduled Task.
The configuration steps are as follows:

  1. Click Start ▶ Settings ▶ Control Panel ▶ Scheduled Tasks.
  2. Create a new Scheduled Task by clicking File ▶ New ▶ Scheduled Task.
  3. Double–click Add Scheduled Task.
  4. On the Tasks tab enter the hvrmaint command line in the Run field, for example:
    c:\hvr\hvr_home\bin\hvrmaint.exe 
    c:\hvr\hvr_config\files\hvrmaint_hourly.opt
    
  5. On the Schedule tab, configure when the hvrmaint script should run.
  6. When ready click the OK button.
  7. A dialog now appears requesting Windows account information (username and passwords). Enter this information as requested.

A sample option file for hourly monitoring could be:

–hub=hvr/!{s8Dhx./gsuWHUt}!			# Encrypted Oracle password 
–sched_option=-h oracle
–env=HVR_HOME=c:\\opt\\hvr_home
–env=HVR_CONFIG=c:\\opt\\hvr_config
–env=HVR_TMP=c:\\temp
–env=ORACLE_HOME=c:\\distr\\oracle\\OraHome817
–env=ORACLE_SID=ORA817
–email_from=hvr@prod.mycorp.com
–email=bob@mycorp.com;jim@mycorp.com
–email_only_when_errors
–snmp_server=snmp.mycorp.com
–output=c:\\opt\\hvr_config\\files\\hvrmaint.log
–scan_hvr_out<br /><br />

A sample option file for weekly restart of HVR on Windows would be:

–hub=hvr/!{s8Dhx./gsuWHUt}!				# Encrypted Oracle password 
–sched_option=-h oracle
–env=HVR_HOME=c:\\opt\\hvr_home
–env=HVR_CONFIG=c:\\opt\\hvr_config
–env=HVR_TMP=c:\\temp
–env=ORACLE_HOME=c:\\distr\\oracle\\OraHome817
–env=ORACLE_SID=ORA817
–email_from=hvr@prod.mycorp.com
–email=bob@mycorp.com;jim@mycorp.com
–email_only_when_errors
–snmp_server=snmp.mycorp.com
–output=c:\\opt\\hvr_config\\files\\hvrmaint.log
–scan_hvr_out
–stop
–archive_files=hvr.out				# Only archive log file hvr.out
–archive_keep_days=14		 	 	# Delete files after 2 weeks
–journal_keep_days=4
–start

Sample Output

From: root@bambi.mycorp.com
To: bob@mycorp.com; jim@mycorp.com
Subject: hvrmaint detected 7 errors (323 rows in fail tables) for hub hvr/ on bambi
2017-11-01T21:00:01-06:30 hvrmaint: Starting hvrmaint c:\tools\hvrmaint.opt –hub=hvr/ –stop –start
2017-11-01T21:10:21-06:30 hvrmaint: Stopping HVR Scheduler 4.4.4/5 (windows–x64–64bit).
2017-11-01T21:10:33-06:30 hvrmaint: Scanning d:\hvr_config\log\hvr\hvr.out (2017-11-01T21:00:03-06:30).
2017-11-01T21:11:13-06:30 hvrmaint: 7 errors (323 rows in fail tables) were detected during scan.
2017-11-01T21:12:33-06:30 hvrmaint: 3 capture jobs for 1 location did 606 cycles.
2017-11-01T21:12:59-06:30 hvrmaint: 6 integrate jobs for 2 locations did 400 cycles and integrated 50 changes for 3 tables.
2017-11-01T21:13:53-06:30 hvrmaint: Archiving 9 log files to d:\hvr\archivelog\hvr_20050209.
2017-11-01T21:16:23-06:30 hvrmaint: Purging 0 archive directories older than 14 days.
2017-11-01T21:18:29-06:30 hvrmaint: Starting HVR Scheduler 4.4.4/5 (windows–x64–64bit).

–––––––––––––––––––– Summary of errors detected during scan–––––––––––––––––
F_JD1034_RAISE_ERROR_P3 occurred 6 times between 2017-11-01T19:43:52-06:30 and 2017-11-01T20:14:24-06:30
F_JJ106E_TIMEO_DB occurred 1 time at 2017-11-01T21:10:03-06:30

–––––––––––––––––––––––––– Errors detected during scan–––––––––––––––––––––––––––
2017-11-01T21:10:03-06:30: hvrscheduler: F_JJ106E_TIMEO_DB: Wait for response from database slave exceeded the 600 seconds defined by 'timeo_db' attribute. Waiting...
2017-11-01T19:43:52-06:30: channel–cap–d01: F_JD1034_RAISE_ERROR_P3: Error as raised by user during pl/sql procedure statement on Oracle SID.

––––––––––––––––––––––– End of errors detected during scan –––––––––––––––––––––––
2017-11-01T21:19:01 hvrmaint: Sending e–mail to bob@mycorp.com; jim@mycorp.com