Configuring secure, sovereign AIOps for IT teams in regulated sectors with OpManager and Ollama

AI adoption is part of the new world order. Whether you're making a deposit in your bank, shopping online, or booking a cab, chances are there's an AI involved in the process.

"CEOs in our survey say 25% of operational decisions are made by AI without human intervention. But by 2030, CEOs expect the share of operational decisions made by AI to nearly double to 48%" ~ Source: IBM

AI adoption in ITOps, or AIOps.

AIOps or Artificial intelligence in IT operations is a technology that arose from the adoption of AI technologies in IT tools. AIOps platforms incorporate artificial intelligence and machine learning as a fundamental part of their operational workflow.

AIOps is particularly relevant for monitoring tools. Monitoring tools collect telemetry from IT systems, detect anomalies, and generate alarms. However, the sheer telemetry volume and architectural complexity of modern IT systems have made this tougher. AIOps simplifies this with the following advantages:

Reducing noise and false positives from telemetry data and finding the real issues
Analyzing network behavior and flagging subtle anomalies that are harder to detect manually
Finding the real root cause of issues when multiple systems fail with event correlation
Forecasting future network trends based on past data
Summarizing fault events and system health analysis to speed up remediation
Generating insights and troubleshooting steps

Many monitoring and observability platforms have adopted AIOps into their operational workflows. However, most vendors use AI providers that are based on the cloud. This is not ideal for IT teams in some sectors.

Some sectors are 'left out' from AIOps

For organizations in the government, defense, healthcare, and BFSI (Banking, Financial Services, and Insurance) sectors, data is not just an asset; it's a matter of national, financial, or personal security. Shipping this data into the public cloud poses compliance and security risks. AI technologies are also notoriously volatile when these sectors need stable and consistent technologies to run critical ops. Here are some of the risks and constraints they face with AI technologies:

Data exfiltration risks: Public endpoints are highly targeted. Attackers use prompt injection to trick public LLMs into revealing previous session histories.
Loss of control & telemetry: Procurement and IT security teams cannot enforce internal Role-Based Access Control (RBAC) on external platforms.
Data residency regulations: Compliance standards like DORA (Digital Operational Resilience Act), and CJIS (Criminal Justice Information Services) could be violated when shipping data into the public cloud.
AI policy changes: Cloud based AI providers often make unexpected changes in price, usage, privacy policies, capabilities, and access.

Only 23% of IT leaders are very confident in their organization's ability to manage security and governance components when rolling out GenAI tools ~ Source: Gartner

Enter ManageEngine OpManager, sovereign AIOps for secure sectors

OpManager is a network and infrastructure monitoring tool with powerful AIOps capabilities. Unlike other SaaS vendors in the monitoring and observability market, you can deploy it on-premises. Most importantly, OpManager integrates with a local AI runtime like Ollama. Integrating OpManager with Ollama helps IT teams achieve truly air-gapped, sovereign AIOps.

Integrating OpManager with Ollama offers the following advantages:

No data ever leaves the network.
All costs are up-front, and straightforward, with no AI token cost anxiety.
Network latency issues will not affect the AIOps functionalities.
IT teams get full control over system prompts, temperature configurations, and custom context injection.

How to configure the OpManager-Ollama integration

Let's take a look at the configuration steps for OpManager's Ollama integration.

Phase 1: Ollama setup

You have to deploy the Ollama runtime on a machine accessible by your OpManager server.

1. Download and install the runtime

Navigate to the official Ollama download page and grab the binary or installer package tailored to your host operating system (Windows, Linux, or macOS). Run the installer on your target machine.

2. Model selection and hardware allocation

Ensure your host machine has adequate compute and memory resources. While running entirely on a CPU is supported, leveraging a dedicated GPU significantly optimizes token processing and inference speeds. Allocate resources based on your chosen model size:

Entry-level Models (3B–4B): Minimum 4 GB RAM (8 GB recommended).
Mid-range Models (8B–9B): Minimum 12 GB RAM (16–24 GB recommended).
Enterprise / High-end Models (20B–70B): 32 GB to 48+ GB RAM.

3. Pull the local LLM weights

OpManager uses gpt-oss:20b as its default model architecture. If your hardware meets the 20B requirements, pull the default weights. Otherwise, you can pull a highly optimized mid-range alternative like llama3.

Open your terminal or command prompt and execute:

ollama pull gpt-oss:20b

(Alternatively, substitute with ollama pull llama3 if resources are constrained).

Verify that the model weights have been successfully initialized on your local disk:

ollama list

Phase 2: Establish the integration in OpManager

With your local AI actively running, you can now instruct OpManager to route its telemetry prompts to your private endpoint.

Log into your OpManager Web Client console using an administrator profile.
Navigate to Settings >> General settings >> Integrations >> Ollama. Click on configure.
Complete the required configuration parameters:

Organization Name: Input your enterprise or organizational identifier.
Base URL: Enter the network path of your Ollama deployment (e.g., http://localhost:11434 or http://<Ollama-Server-IP>:11434).

Read through the explicit data privacy and local compliance notice on screen, then check the acknowledgment box.
Click Save to finalize the underlying API connection.

Phase 3: Redirect the default AI model (Optional)

If you downloaded an alternative model like llama3 during the installation phase instead of the default gpt-oss:20b, you must modify OpManager's backend parameters to point to the correct model identifier.

Access your underlying OpManager server file system.
Navigate to the core configuration directory: <OPM_HOME>/conf/OpManager/
Open the serverparameters.conf file using a plain text editor.
Scroll to the bottom of the file and append the following configuration parameter line:

OLLAMA_GENAI_MODEL <your-model-name>

Example for mapping to Llama 3:

OLLAMA_GENAI_MODEL llama3

Save and close the configuration file.
Restart the OpManager service to reload the updated server parameters into active system memory.

What can you do with this integration?

Once the handshake between OpManager and Ollama is established, your IT team can immediately leverage local intelligence across five critical workflows without a single byte of data leaving your perimeter:

Active alarms summary: OpManager generates alarms if your systems aren't behaving as expected. With summarization, you get a quick overview of everything that's wrong in your environment.
Alarm summary and analysis: You can drill down into any individual alarm and analyze the raw system telemetry to get immediate context, potential root-causes, and troubleshooting steps.
Asset group summary: If you've created groups to manage your IT assets in OpManager, you can perform quick operational health checks of your network assets as a whole.
IT asset summary: Drill down into any IT asset and instantly translate real-time performance graphs and metrics into a clear text narrative.
Automated script generation: Monitor any custom metric by generating monitoring scripts within OpManager's interface with AI. Describe your requirements in a natural language prompt. The local model will instantly draft ready-to-run monitoring scripts in PowerShell, Python, or Shell.

What are the AIOps features available in OpManager?

Beyond local LLM integrations, OpManager also enables other AIOps features to simplify your IT operations.

Built-in AI assistance: OpManager has Zoho's Zia AI built into its interface. Zia's machine learning (ML) engine can help you run system resource forecasts, generate monitoring insights.
Model context protocol (MCP) server: OpManager supports a native MCP server. You can use it's natural language interface to bring your monitoring stack, helpdesk, DevOps tools and coding assistants under one console.
Cloud-based GenAI integrations: You can also integrate with public AI providers like OpenAI and DeepSeek to get real-time IT insights.

Start your AI sovereignty journey today!

OpManager offers a free thirty day trial. Get in touch with our product experts to see how your IT team can implement secure and sovereign AIOps!