Reference answer
S – Situation About eight months ago, we began receiving numerous complaints from users across multiple departments about slow performance when accessing and interacting with our ServiceNow instance. Specifically, users reported long load times for incident and change request forms, slow searching within list views, and general sluggishness when navigating between modules. This wasn't isolated to a few users or specific times; it was a pervasive issue impacting daily operations for hundreds of users. The increased wait times were causing frustration, reducing productivity, and sometimes leading to users having to refresh their browsers multiple times, which sometimes meant losing unsaved work. Our internal SLAs for ticket resolution were also being impacted due to the delays in accessing and updating records.
T – Task My primary task was to thoroughly investigate these performance complaints, identify the root cause(s) of the performance bottleneck, and implement effective solutions to restore optimal system responsiveness. This was a critical task because pervasive slow performance can erode user trust in the platform and ultimately lead to decreased adoption and dissatisfaction. I needed to act methodically, gather data, analyze it, and apply appropriate technical remedies without causing further disruption. The goal was to significantly reduce form load times and improve overall navigation speed across the platform.
A – Action
I began by leveraging ServiceNow's built-in diagnostic tools. First, I activated Stats.do
and Debug Trace
for myself and then for a few affected users to capture detailed performance metrics, including server response times, network latency, and client-side processing. This quickly showed consistently high server response times for specific forms and list views, indicating a server-side bottleneck rather than just client-side or network issues.
Next, I delved into the System Diagnostics
module, specifically looking at Transaction Log
(syslog_transaction
) and Slow Queries
(sys_query_history
). I filtered the Transaction Log
by duration, focusing on transactions exceeding 5 seconds. This immediately highlighted that loading the incident.do
and change_request.do
forms, as well as queries on the cmdb_ci
table, were consistently among the slowest operations. The Slow Queries
log pointed to several complex database queries being executed by specific business rules and related lists on these forms.
Further investigation into the identified slow forms revealed a few culprits:
- Overly Complex Business Rules: On the Incident form, a business rule was querying multiple related tables (
sys_user
,cmn_location
,task_sla
) ondisplay
to dynamically populate user information fields and calculate SLA targets, even when not strictly necessary. This was executing every time the form loaded. - Unoptimized Related Lists: The Change Request form had several related lists displaying information from tables with millions of records (e.g.,
sys_audit
,workflow_context
). These related lists were loading all records by default instead of using appropriate filters or pagination, leading to massive database queries. - Client Scripts with DOM Manipulation: Some legacy client scripts were performing extensive DOM manipulation on form load, which added to client-side rendering delays.
- Inefficient Data Imports: A scheduled daily import for CI data into the
cmdb_ci
table was taking an excessively long time and locking tables during its execution, contributing to overall system sluggishness.
My resolution actions were multi-faceted:
- Business Rule Optimization: I refactored the problematic Incident business rule. Instead of querying multiple tables on
display
, I modified it to useasync
business rules where possible, and only fetch data when specific fields were modified or when a user explicitly requested it (e.g., through a UI Action). I also ensured that queries within business rules were using appropriate indices. - Related List Configuration: For the Change Request form, I modified the
Configure > Related Lists
settings. I changed related lists displaying large datasets to use reference fields or simpler queries, or in cases where historical data was needed, I configured them to load only a limited number of records by default, with an option to "View All." I also ensured appropriate indexes were on the reference fields. - Client Script Review: I audited the client scripts on the slow forms. I identified and deprecated several old client scripts that were performing unnecessary DOM manipulations or making synchronous GlideAjax calls, replacing them with more efficient
g_form
methods or asynchronous calls. - CMDB Import Optimization: I worked with the team responsible for the CMDB import. We optimized the transform map, added coalesce fields, and implemented a more efficient import set strategy using multiple import sets and smaller batches, which significantly reduced the execution time and prevented table locks during peak hours.
- Database Indexing: Based on the
Slow Queries
log, I identified a few custom tables that were frequently queried without proper indexing. I worked with our database administrator to add appropriate indexes to these tables, which dramatically improved query performance for those specific modules.
R – Result
The combination of these actions led to a dramatic improvement in ServiceNow performance. Form load times for Incident and Change Requests decreased by an average of 60%, from around 10-15 seconds down to 3-5 seconds. List view searches became near-instantaneous, and general navigation felt significantly snappier. User complaints about slow performance virtually ceased, and feedback shifted to praise for the renewed responsiveness of the platform. Our internal ticket resolution SLAs returned to acceptable levels. The project not only resolved the immediate performance issues but also instilled a stronger culture of performance-aware development and configuration among our team. We now regularly review Slow Queries
and Transaction Logs
as part of our health check routine, preventing similar issues from recurring and ensuring a consistently fast and reliable ServiceNow experience for all users. This experience solidified my ability to diagnose complex platform issues and implement holistic solutions, rather than just superficial fixes.