Skip to main content

ORA-00800: soft external error arguments

The problem as stated:
After upgrading the Oracle Database from 12c to 19c the following error generated in the alert log file.

Actual Oracle Error Code: ORA-00800
After Upgrading from 12c to 19c ORA-00800: soft external error, arguments occured
Oracle Database Enterprise 19c fails to start with error: ORA-00800: soft external error, arguments: [Set Priority Failed], [VKTM]
Why does Insights prevent Oracle grid services from starting
How to configure the User slice to allow Real-Time Scheduling for user processes

In Alert Log:

Errors in file /u01/app/oracle/diag/rdbms/orclproddbone/orclprod/trace/psbcdb_vktm_3529.trc (incident=1200048) (PDBNAME=CDB$ROOT):
ORA-00800: soft external error, arguments: [Set Priority Failed], [VKTM], [Check traces and OS configuration], [Check Oracle document and MOS notes], []
Incident details in: /u01/app/oracle/diag/rdbms/orclproddbone/orclprod/incident/incdir_1200048/psbcdb_vktm_3529_i1200048.trc
2023-10-30T10:04:53.536165+06:00
Error attempting to elevate VKTM's priority: no further priority changes will be attempted for this process

In Trace File:

more /u01/app/oracle/diag/rdbms/orclproddbone/orclprod/trace/psbcdb_vktm_3529.trc

[oracle@dbupgradesrv ~]$ more /u01/app/oracle/diag/rdbms/orclproddbone/orclprod/trace/orclprod_vktm_3529.trc
Trace file /u01/app/oracle/diag/rdbms/dbupgrade/orclprod/trace/orclprod_vktm_3529.trc
Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.3.0.0.0
Build label: RDBMS_19.3.0.0.0DBRU_LINUX.X64_190417
ORACLE_HOME: /u01/app/oracle/product/19c/db_1
System name: Linux
Node name: Server
Release: 4.18.0-477.10.1.el8_8.x86_64
Version: #1 SMP Wed Apr 5 13:35:01 EDT 2023
Machine: x86_64
Instance name: orclprod
Redo thread mounted by this instance: 0 <none>
Oracle process number: 5
Unix process pid: 3529, image: oracle@Server (VKTM)

*** 2023-10-30T10:04:53.535009+06:00
*** CLIENT ID:() 2023-10-30T10:04:53.535027+06:00
*** SERVICE NAME:() 2023-10-30T10:04:53.535031+06:00
*** MODULE NAME:() 2023-10-30T10:04:53.535034+06:00
*** ACTION NAME:() 2023-10-30T10:04:53.535037+06:00
*** CLIENT DRIVER:() 2023-10-30T10:04:53.535042+06:00

2023-10-30T10:04:53.534502+06:00
Incident 1200048 created, dump file: /u01/app/oracle/diag/rdbms/orclproddbone/orclprod/incident/incdir_1200048/orclprod_vktm_3529_i1200048.trc
ORA-00800: soft external error, arguments: [Set Priority Failed], [VKTM], [Check traces and OS configuration], [Check Oracle document and MOS notes], []

*** SESSION ID:(2351.6983) 2023-10-30T10:04:53.570556+06:00
*** SERVICE NAME:() 2023-10-30T10:04:53.570568+06:00
*** CONTAINER ID:(1) 2023-10-30T10:04:53.570572+06:00

kstmmainvktm: failed in setting elevated priority
Verify: SETUID is set on ORADISM and restart the instance
highres_enabled
VKTM running at (100ms) precision
kstmrmtickcntkeeper: param _dbrm_quantum will not be effective
[Start] HighResTick = 209726095
kstmrmtickcnt = 0, ksudbrmseccnt[0] = 1698638693
[oracle@Server ~]$

Root Cause According to Red Hat: 

1. Short answer: The insights-client service unit sets a CPU quota of 30% on the system.slice
             /usr/lib/systemd/system/insights-client.service
2. Long answer:
On RHEL servers systemd is used to control services.
These services can ask for CPU quota.
In order to provide this CPU accounting, systemd creates the CGroup nodes for system.slice and user.slice, but it cannot set the value of the cpu.rt_runtime_us quantum.
This effectively disables realtime scheduling for all processes assigned to that slice
The system.slice is affected here.
The Oracle Grid software relies on the realtime scheduling.
There currently is no other effective way to limit CPU.

Diagnostic Steps According to Red Hat: 

1. The /var/log/messages log file contains:
Mar 31 02:53:36 example-grid-srvr1 kernel: Initializing cgroup subsys cpuset
Mar 31 02:53:36 example-grid-srvr1 kernel: Initializing cgroup subsys cpu
Mar 31 02:53:36 example-grid-srvr1 kernel: Initializing cgroup subsys cpuacct

2. Confirm changes on the system made by the insights-client
# find /etc/systemd/system.conf /etc/systemd/system /usr/lib/systemd -type f | xargs grep -e CPUAccounting -e CPUWeight -e StartupCPUWeight -e CPUShares -e StartupCPUShares -e CPUQuota |grep -v -e :# -e "^Binary file"
/usr/lib/systemd/system/insights-client.service:CPUQuota=30%

3. Removal of the insights-client package allows the Grid services to start

My Step:

  1. Remove insights-client package: [root@Server ~]# dnf remove insights-client
  2. Shut Down Database
  3. Reboot Operating System
  4. Open Alert Log: [oracle@Server ~]$ tail -f -n 22 /u01/app/oracle/diag/rdbms/orclproddbone/orclprod/trace/alert_psbcdb.log
  5. Start Database Gradually: nomunt -> mount -> open -> open pluggable database and frequently check Alert Log
  6. The error ORA-00800 is gone.

References: 

https://access.redhat.com/solutions/6991342
https://access.redhat.com/solutions/6020561
https://access.redhat.com/solutions/6991342
https://access.redhat.com/solutions/7004058



 
Labels:

Comments

Post a Comment

Popular posts from this blog

Upgrading Issue for RHEL 7 to 8 With Leapp

Overview The Leapp utility is a framework for updating and upgrading operating systems as well as applications. The operations of this utility consist of two phases 1. the preupgrade Phase – that chack the upgrade possibilities and 2. the actual upgrade phase – that map packages between previous and current versions of the software packages. Issue – 01: After running ‘ sudo leapp preupgrade ‘ sometimes you find the below issue in ‘ /var/log/leapp/leapp-report.txt ‘. Detail: Risk Factor: high (inhibitor) Title: Leapp detected loaded kernel drivers which have been removed in RHEL 8. Upgrade cannot proceed. Summary: Support for the following RHEL 7 device drivers has been removed in RHEL 8: – pata_acpi Key: f08a07da902958defa4f5c2699fae9ec2eb67c5b Remediation: 1. Disable detected kernel drivers in order to proceed with the upgrade process using the rmmod or modprobe -r . rmmod – Simple program to remove a module from the Linux Kernel modprobe – Add and remove modules from the Linux Ke...
Post a Comment
Read more

The Oracle Management Server (OMS) host and port specified via OMS_HOST and EM_UPLOAD_PORT is not available

The Oracle Management Server (OMS) host and port specified via OMS_HOST and EM_UPLOAD_PORT is not available. 1.0. error message: Error Message: The Oracle Management Server (OMS) host and port specified via OMS_HOST and EM_UPLOAD_PORT is not available. Pass in a valid hostname and port number for the OMS to successfully deploy the agent. 1.1. deploy log: Log: /opt/gc_inst/em/EMGC_OMS1/sysman/agentpush/2023-11-07_16-09-23-PM/applogs/dblivesrv_deploy.log Execution of command cd /opt/agent/ADATMP_2023-11-07_16-09-23-PM;/opt/agent/ADATMP_2023-11-07_16-09-23-PM/agentDeploy.sh -ignorePrereqs ORACLE_HOSTNAME=dblivesrv AGENT_BASE_DIR=/opt/agent OMS_HOST=oemsrv.localdomain EM_UPLOAD_PORT=4903 AGENT_INSTANCE_HOME=/opt/agent/agent_inst b_doDiscovery=false START_AGENT=false b_forceInstCheck=true -force AGENT_PORT=3872 on host dblivesrv Failed 1.2. oms details: [oracle@oemsrv ~]$ $OMS_HOME/bin/emctl status oms -details Oracle Enterprise Manager Cloud Control 13c Release 5 Copyright (c) 1996, 2...
Post a Comment
Read more