A leading producer maximizes nitrate yield using CONNECT and Databricks

Sociedad Química y Minera de Chile (SQM) is one of the world’s leading producers of specialty plant nutrition, iodine and its derivatives, and nitrate-based industrial chemicals. With over 8,300 employees and approximately $4.5 billion in annual revenue, SQM operates across multiple continents, serving the agriculture, pharmaceutical, health, and sustainable technology industries.

Within SQM, the iodine and plant nutrition division (SQM YNV) manages the nitrate production plants. This is critical to the company’s global supply chain. By improving yield, reducing waste, and optimizing resource consumption, SQM recognized it could scale its sustainability goals, as well as maintain its leadership role in specialty chemicals.

Maximizing nitrate production required operational adjustments

SQM wanted to maximize nitrate production yield by optimally adjusting operational parameters such as feed ratios, temperatures, and flow conditions. Improving yield was critical to increasing operational efficiency, reducing production costs, and protecting margins in a business that remains an important contributor to SQM’s portfolio. Before optimization could begin, SQM needed to address several operational and process challenges.

Decisions were historically based on experience, not on data. Operators relied heavily on intuition and years of accumulated know-how, with no dynamic system suggesting optimal setpoints. For instance, SQM had no automated predictive or simulation models to enable operators to preempt failures before they impacted productivity. These tools would enable the operators to run real-time scenario simulations and adjust variables based on model outputs.

In addition, SQM’s processes involving raw materials were inefficient. For instance, certain conditions led to excessive salt consumption and higher feed/production ratios. The team recognized this could be resolved with dynamic optimization.

To address these operational and process challenges, SQM first had to overcome a series of data and technology obstacles as follows:

• Disconnected data sources: SQM lacked a unified analytics layer marrying process, sensor, and lab analytics data with simulation outputs.
• Limited ability to share operational datasets: SQM needed a more scalable, developer-friendly way to expose operational data to cloud analytics/AI platforms like Databricks.
• No closed-loop workflow: Model results could not be seamlessly fed back into on-prem systems for operator visibility or historical tracking.

It was clear to SQM that they needed an architecture that unified operational data, simulation models, and AI, in order to enable real-time insights and recommendations to improve nitrate yield.

Centralized industrial data backbone

To resolve its challenges, SQM deployed CONNECT atop AVEVA PI System and leveraged the solution as the industrial data backbone, using its point-and-click interface for easy implementation. The company also integrated its OT data via Delta Sharing with Databricks on Azure, making it available in Unity Catalog for simulation, machine learning, and optimization modeling. These solutions enabled a full closed-loop workflow from on-prem operations to cloud analytics and back to the plant, with the following capabilities:

Unified data architecture for analytics

• PI to CONNECT Agent securely transferred AVEVA™ PI Server data from the nitrate plant into CONNECT.
• A curated data view and virtual table was created, containing relevant operational variables using one-minute aggregates.
• Databricks accessed this data in Unity Catalog through standard, open Delta Sharing, refreshed every five minutes.

 Rapid development of ML and hybrid models

• Worked with SQL and Python notebooks to validate data density and data quality.
• Trained and retrained ML and hybrid simulation models to estimate optimal plant conditions.
• Developed advanced dynamic optimization models to generate real-time recommendations.

Closing the loop back to operations

• Model outputs were written from Databricks back into CONNECT, then to AVEVA PI Server using CONNECT to PI Agent.
• Operators could visualize ML model predictions and recommendations in CONNECT’s trending tool and AVEVA visualization tools, including AVEVA PI Vision.
• This enabled operators to compare actual performance with model-guided optimal values.

By using AVEVA PI System, CONNECT, and Databricks, SQM established a digital foundation that transformed operations and accelerated business value. The Lighthouse pilot was executed over several weeks, with operations staff, data scientists, analytics engineers, and CONNECT specialists collaborating to ingest the data, create data views, and develop predictive and optimization models.

This cross-functional team created a digital foundation that transformed operations and accelerated business value:

• Predictive models are now highly accurate (about 90%), helping operators make confident decisions.
• Time to insight dropped from days to minutes thanks to automatic refresh cycles.
• Model retraining was reduced from weeks to hours, accelerating iteration and improving adaptability as process conditions evolve.

The new digital foundation also delivered value to operators:

• Near-real-time (five-minute) updates support continuous optimization cycles.
• Operators reduced manual analysis effort by 30 - 50%, focusing more on adjustments and less on data gathering.
• SQM now has an enterprise-ready, developer-friendly industrial data pipeline to support new analytics initiatives at scale.

"A great value was unlocked when data could be transmitted from (AVEVA) PI System to Databricks, our Data and AI platform. A developer-friendly and industrial solution for moving data at scale is now available for us. Not only this particular use case is achievable, but also many other operational challenges that will arise."

–Matias Gatica, Digital Transformation Manager, SQM YNV
 

Improved nitrate yields and reduced waste

SQM’s new architecture helped the team activate a new, data-driven way of operating its nitrate production processes, delivering business value in numerous ways.

• Nitrate yield increased by up to 1%, driven by optimized process conditions and more consistent high-performance operating windows.
• Raw material efficiency improved by 1 - 3%, reducing salt consumption and minimizing chemical losses.
• Variability in key parameters decreased by 10 - 15%, improving plant stability and reducing off-spec operating periods.

What’s next for SQM?

Building on the success of the nitrate yield optimization project, SQM plans to expand this approach across more nitrate plants and production lines, increasing the impact of data-driven decision-making. The company is exploring richer, multivariable models by integrating lab, maintenance, and quality data, as well as real-time optimization agents to suggest operator setpoints and reduce process variability.

With AVEVA PI System, CONNECT, and Databricks as the enterprise data foundation, SQM is creating a scalable framework to support analytics across its iodine, lithium, and plant nutrition divisions, while investing in internal skills to enable engineers and data specialists to build and refine predictive models.

These initiatives mark the start of a broader transformation, integrating data, models, and operations to enhance plant performance, resource efficiency, and sustainability across SQM’s global operations.