Events like installation of updates, or raid failures can impede the operation of Hardware Appliance services. In such cases, you will not see long error messages. Instead, the Hardware Appliance will be put into maintenance state.

In maintenance state, all access to EJBCA/SignServer over HTTP(S) is disabled. Each request will return a message informing you that the system is in maintenance and cannot be accessed.

Hardware Appliance States

To find out the state of the Hardware Appliance, go to the page Platform > Troubleshooting in WebConf. The Hardware Appliance can be in one of three different states:

  • Operational
    The Hardware Appliance is fully operational. All subsystems are working as expected.
  • Maintenance
    The Hardware Appliance is in maintenance mode and application services are interrupted by an automatically detected reason.
    The state Maintenance can replace the state Offline.
  • Offline
    The Hardware Appliance is in maintenance mode and application services are interrupted by a manual setting in WebConf.
  • The Hardware Appliance can only enter the Offline state when this is set manually and no other automatically detected reason occurs. Any automatically detected reason will change the state from Offline to Maintenance. In such a case, the manual Offline setting is still active but invisible. When all automatically detected reasons disappear, the Hardware Appliance enters the Offline state again.
  • Most customers will not see any difference between the Offline and Maintenance state. Operators, however, will know that the Offline state indicates a maintenance mode that is manually set in WebConf. In contrast, the automatical Maintenance state indicates a real problem. Here, the Hardware Appliance services were not taken offline by choice.
  • It makes sense during an automatically induced Maintenance state to also set the Hardware Appliance manually Offline in WebConf: Operators can then check the integrity of the Hardware Appliance after an incident before exposing services to customers.

Reasons for Maintenance state

The Hardware Appliance will be put into Maintenance state automatically for the following reasons:

  • Factory Reset During Operation
    The Hardware Appliance will be set to Maintenance automatically if the Factory Reset procedure is triggered during operation. The Maintenance state ends with the next reboot finishing the Factory Reset. This event is not recoverable.

  • RAID Failure
    The Hardware Appliance will be set to Maintenance automatically if a fatally broken RAID is detected. The reason behind this:
    The Hardware Appliance will enter an inconsistent state if both SSD hard disk drives fail. This state cannot trigger any error messages until caches are finally flushed. Setting Maintenance with one broken RAID ensures that no data is created that cannot be recovered later.
    This event is not recoverable.

  • HSM Alarm
    The Hardware Appliance will be set to Maintenance automatically if the embedded HSM has detected an alarm. Due to the alarm, all key materials will be erased by the HSM. Running EJBCA/SignServer without a working HSM therefore does not make any sense.
    This event is not recoverable.

  • Database is Down
    The Hardware Appliance will be set to Maintenance if the embedded database system stops operating. When the database is available again Maintenance state is stopped automatically.
    This event is recoverable.

  • Application Update
    The Hardware Appliance will be set to Maintenance if an application is updated via WebConf. When the update has finished Maintenance state is stopped automatically.
    This event is recoverable.

  • Manual Setting 'Offline'
    You can use the Offline function in the WebConf page Platform > Troubleshooting to manually activate the maintenance mode for the Hardware Appliance. In contrast to the automatic Maintenance state, the Offline state is not based on any apparent problem/cause. You can use this function to disable customer access to EJBCA/SignServer without completely shutting down the Hardware Appliance. Customers will then see the Notification page that is described below.

The Offline state will not persist a reboot of the Hardware Appliance.

Effects of the Maintenance state

The following sections describe changes and information shown when the Hardware Appliance is operating in maintenance.

Notification Page

Every HTTP(S) request to EJBCA/SignServer will lead to an HTTP 501 status code response. A web page appears and notifies the user that the Hardware Appliance is currently not operational and running in maintenance.

OCSP requests will also receive an HTTP 501 status code with that notification page inside the responses body.

Front Display

When the Hardware Appliance enters maintenance the messages on the front display include the following:

MAINTENANCE
Services unavail

The message disappears when the state of the  Hardware Appliance switches back to operational.

WebConf

Troubleshooting Section

In the WebConf page Platform > Troubleshooting all maintenance reasons will be listed.

If the Hardware Appliance is set to Offline this will only be reflected by a change in the button: the button name Offline switches to Online.

Warning Messages

When a WebConf page is opened during maintenance the white-on-red message Services Unavailable appears in the upper left corner. After leaving maintenance, the message will disappear when the page is reloaded or when a new page is opened.

SNMP

If SNMP is enabled it will indicate the Hardware Appliance state. it will also show a human-readable combined message of all reasons for maintenance. For more details refer to section Monitoring > SNMP.

Syslog

Syslog and AVM server log will contain detailed messages about changing events that lead to state changes of the Hardware Appliance.

Support Package

Each time the Hardware Appliance enters maintenance a Support Package will be created. This also happens if the Hardware Appliance has been set Offline manually.

If the Hardware Appliance is already in maintenance state, no additional Support Package will be created. For example, if the SSD harddisk drives all fail and the Hardware Appliance is automatically set to the Maintenance state. Minutes later the Factory Reset is triggered. In such a case, only one Support Package will be created. For more information, see Support Packages.