Welcome to the mining metaverse

Joe Renshaw

Rebecca.

Rebecca Ahrens

Joe.

Joe

Close your eyes, picture a mine, and tell me what do you see.

Rebecca

Honestly, the first thing that comes into my head is the original, animated Snow White and Seven Dwarves. I think that was probably my first ever exposure to a concept of a mine. Is that the wrong answer? [laughter]

Joe

No, that’s a perfectly legitimate answer! Where I’m going with this is that, apart from the small number of people who have actually worked on a mine, no one really knows what a mine looks like or how it works.

And actually, even for people who do know, in theory, how a mine works, it can be really hard to keep track of what’s going on.

Rebecca

Right. Because when you’re on the site of a mine, everything is changing all the time: trucks, equipment, even the literal ground beneath your feet.

Joe

Yeah. But technology—and specifically digital twins—have the potential to profoundly clarify the inner workings of a mine.

So to learn more about how mines actually work, and how digital twins can help them work better, we’re going to be talking to a gentleman by the name of Preston Miller.

Rebecca

Preston is a veteran of the mining industry. He’s worked in mining for decades both in the field as a mining engineer and on the business-improvement side. He’s also built digital twins of mines. So he’s really perfectly placed to help us update our mental model of a modern mine—from animated dwarves pick-axeing...whatever they were pick-axeing: I don’t even remember. Was it diamonds? I feel like it was diamonds.

Joe

[laughing] I can’t remember.

Rebecca

Coal? Anyway, from that to what a modern, tech-fueled mine really looks like.

Joe

But before we jump into our conversation, an announcement: we have a fancy new website! Head to www.ourindustriallife.com to find tons of stories about the industrial sector, sortable by topic and industry, so you can find exactly what you’re most interested in.

And over there you can also sign up for our fortnightly newsletter.

Preston Miller: From drill and blast to data visualization

Rebecca

Preston Miller, welcome to our industrial life. I'd love to start off with just having you talk a little bit about your background, your experience in mining, how you got into the industry, what sort of projects you’ve worked on over your career.

Preston Miller

Yeah, so, I'm originally born and raised in Alaska, and I've loved mineral exploration since I was a kid. I was one of those kids with a 500-pound rock collection. And my father worked in the mining industry for years and years. And so when I graduated high school it was kind of an obvious choice to go into mining.

I’ve also been very much into visualization as a tool of helping people understand—including myself—what's going on inside mines. So, as a mine engineering student, there was one class that I had—underground mining methods—where I took the Doom video game engine and used it to demonstrate what several underground mine methods look like. 

And that kind of started my journey in using digital technology in understanding and facilitating the learning about mining. When I graduated, my first role out of school was drill and blast engineer. And it was great. I got exposure to production mining. And then mine planning and then longer-term mine planning, out to strategic, life-of-asset, closure planning. And after doing that for a while, I became very much interested in continuous improvement.

And that's when my career really started to take a shift into digital applications, advanced reporting, advanced analytics, then eventually 3D visualization, extended reality, augmented reality for use in mining. And that kind of leads me to where I am today.

How a mine works

Joe

Well, I am someone who's never visited a mine before. So, perhaps you could explain how a mine works and some of the, kind of, inefficiencies that what you're describing—this kind of interactive model—would help kind of resolve. 

Preston

So, at a very high level mine operations are typically split into production mining operations, where you're moving material and then you're grinding it down and you're feeding it into a mill to concentrate the valuable—say it's copper or zinc or lead. So you've got both mobile and fixed assets and on both of those you have operations and maintenance. And then between those two you have typically day shift and night shift. Right? And so you've got these different moving parts that need to know where each of the other ones are. 

And it needs to be done in a near real-time interactive state in order for people to get the information they need to make decisions immediately, to maximize the value from production and maintenance operations. 

So you can imagine that if everything is done via static reports that come from Excel, it takes a huge amount of time just to prepare those reports. And it takes a long time, you know, compared to if you've got your reporting systems tied in near real time.

Rebecca

Can you give a couple examples of a type of decision that needs to be made quickly, and the more quickly it's made, the better for the outcome? So, something that: if you got a static report—you had to read through a list of numbers and then make a call—versus seeing in real time: oh, I can visually see the situation that you're talking about, whether it's a malfunction with a machine or a status of a load that's coming in. 

Can you talk through an example like that? 

Preston

Absolutely. So in my time as a more junior mining engineer, one of the things that I did was field supervision. And shift change is a very—it's a challenging time because you're trying to figure out what happened in the previous shift, where all the equipment is, what's the status of the equipment, and what you're going to need to do to dispatch your crew in the most efficient manner. 

So, in the older days, the way it works is you would get notes from the previous shift supervisor. And there would be a table. They would fill out the equipment. They would list where it was to the best of their knowledge. What's the fuel level on it? What's been fueled? What hasn't been fueled? What equipment's recently gone down. And they would tell you what equipment has come up so you can very quickly at the beginning of the shift dispatch your personel in the most efficient manner. 

Rebecca

And they're doing this from memory to some extent, right? Because they're not—it's a large space that they're in—so they're like, last I checked, this piece of equipment had this status, but I haven't been over there physically in, whatever, 15 minutes. 

Preston

That's right. And on top of that, the supervisor is responsible for physically going out and inspecting each of the areas of work. 

So you can imagine it's a very busy time and you're juggling a lot of balls trying to get your workforce out there and being productive as quickly as possible. I’m talking about that from the lense of a mine field production supervisor, but it extends to all the different supervisory activities.

How to build a digital twin of a mine

Rebecca

Last summer when we first spoke you were working for Teck at the time and I know you were working on a pretty interesting project there. Are you able to talk about the projects that you’ve done—even if they were just, you know, kind of a proof of concept in your previous roles?

Preston

So, back when the pandemic happened and everybody ended up working remotely 100% of the time, I was going stir crazy. In Alaska, we call it cabin fever. I was able to get my hands on a HoloLens 2 and then started learning how to develop for HoloLens 2 using Unity Game Engine. 

So what I did I built out a prototype for an augmented reality application that assisted heavy equipment maintenance personnel. The idea was to not only give them the ability to call in somebody through remote assist, but also to help track and guide their maintenance activities to help them understand, you know, what is the maintenance background on this piece of equipment? What is our parts inventory look like? Are you gonna have the parts to complete the maintenance operation? And if not, when are we expecting those parts, right? So after building a prototype, I was able to pitch the creation of the application to an internal program. And then we built it and deployed it.

And it worked and it really opened a lot of eyes to  possibilities that emerging technology could provide, especially when considered that you could integrate our existing reporting and data systems with it. So that was really my first venture into extended reality. 

The thought was always there: eventually this is going to turn into something metaverse-like. I know that everybody has read books on the metaverse and they've heard things. Most people still struggle to understand what it really is and what it can do.

I was able to prototype out and with a couple other people in the company.
 

Rebecca

And this is still at Teck. 

Preston

Still at Teck Resources, a very high-fidelity 3D virtual environment that was integrated with a number of our data systems including our fully automated drone data processing pipeline.

So we were able to have a very realistic-looking 3D environment, and then connect it to our data systems reporting and particularly our mobile equipment data so that we could stream in the locations and instantiate 3D models representing the equipment, showing where it was with respect to the latest topography in the mine.

Rebecca

Just to clarify: so the drone data is giving you the location, say, of a particular truck, but you're also pulling telemetry off of that truck to understand how the various components of the truck might be working. So you're getting both of those streams fed into this sort of immersive visualization environment. 

Preston

Yes. So, what the drones provide is the latest topography. So the surface meshes. Mining operations out in the pit are very dynamic in terms of material movement. So where material is being brought from, from the dig face, where it's being dumped: it's changing all the time. The roads stay relatively static, but if you want to know what's the latest state of your mining operation, you really need those drones flying very regularly.

And so we had built out an enterprise-wide drone program so that all of our drone pilots—who were typically surveyors at the time—were able to use standardized equipment and automated cloud processing workflow. And then we built out a system where we had what we called a master mesh. So as new information was coming in from the drone flights, it would be built in to the master view of what was going on at the mine site. So that would provide what the ground truth is at the mine.

And then the mobile equipment data was something different. It was coming off of—actually we have multiple systems. And so what we did is tapped into that data and got near real-time views of where the equipment was. And then we used the magic of Unreal Engine to animate the trucks, show where they're at,  where they're getting loaded, where they're dumping material.

And then we were also able to get the BIM data from our milling operations and then get that textured and bring that in. And of course it was all using a real-world coordinate system. So as you were seeing things happen, it's where they really were in the real world.

And when you see something like that take shape: imagine a very advanced video game, but what you see happening in the video game is what's actually happening in the real world. We were noticing that there's a lot of power there. 

And the power comes from having a real-time ground truth understanding of what's happening at the mining operation: a God's eye view, so to speak. 

Rebecca

And when you say a video game, I mean, you mean a user could go in, create a character, like you can in a video game, and then walk around in the virtual environment that represents the most up-to-date version of what the mining environment looks like. 

Preston

Exactly—and integrated with our business reporting. So you can instantiate 2D browsers and then call in your Power BI reports for any particular area. And then on top of that—so yes, you could create an avatar—but it was also a multiplayer. So you can imagine: you can meet somebody in this environment and go look at things and talk about them.

So it wasn't just people at the head office or the mine office who could see what was going on out in the field. It would eventually be something that people can use out in the field on a smartphone, a tablet, or even a headset later on to be able to see an overlay of all the IoT data that's coming off of our mining equipment, whether it's mobile or fixed. So, imagine that, but then being able to use that as a place where people from the office and the field can collaborate and see essentially the same thing. It's pretty amazing. 

Return on investment of Digital Twins

Joe

I've got question about, I guess, the break-even point of a project like this for a mine. I mean, I know there are lots of different mines with different lifespans and all that. But you obviously have to put quite a lot of effort into the development of a digital twin of a mine. Every digital twin is different, obviously, that's whole point. So, what's your sense of the return on investment for a project like the one you're describing? 

Preston

Okay, so yeah, let's back up just a little bit and talk about digital twins in general and why they haven't taken off to the extent that we've expected them to in the past. Everybody has their theories. My theory is that digital twins have been siloed. They've been built for their own specific area and the only people who can see them are the subject matter experts for that particular area. 

The approach that we took was to create a unified environment where all these digital twins exist in, where they exist in the real world and connected to their real-world data. And when you do that and you make it scalable, you're not thinking about just making a digital twin of one mine. You're talking about making a digital twin of all of your mining operations. 

So for example, for Teck Resources, we've got quite a number of mines in both North and South America. And so when you enter this unified digital twin, you start off at the globe level. It looks like Google Maps. 

Of course, you've got the UI and all that, but you zoom in to see this mine over here. And then you zoom in to the particular area that you're interested in, whether it be milling, mining, material transport, export—and being able to see it all in one place.

And when you build the system with the ability to expand very quickly, then it starts generating the question: how unified are your data systems? That's probably the most important thing, is having your data systems in place and interoperable, and having a plan where you're going to unify this for visualization and interaction.

And so if that's done well, it can actually be done for—I don't want to say relatively cheap—but look at the amount of money that mining organizations spend on software, and you're talking about a fraction of the overall budget for all the licensing, for all of the software. 

And so I think the return is absolutely there. And then the question becomes: how do you quantify the value for something like this?

Rebecca

Always the question, right?

Preston

It's difficult to quantify the value. So we have to look back at how have we quantified the value before. OK. So in the past, we used Excel, connected to databases, or sometimes, many times, the data is copied and pasted in and it takes hours to build these reports.

Well, when you use a tool like Power BI that has an automatically refreshing connection to your data systems, you build the report, the report refreshes itself, right? So you go from one or more engineers time, hours per day for one particular report down to: no, you just pull the report up, it's refreshed automatically, right? And then thinking about that as the extension of where we're going in industrial metaverse, now we're talking about the larger scale reports that require information from so many different sources that take all this time.

Once you build it out and you've got integration between all these different data sources, you continue saving time in generating the information that's needed to make better decisions. So that's one way to go about it.

Now, when you look at the actual impact: people making better decisions with the data in a more timely manner. That is, that's extremely difficult to quantify. So there's a leap of faith involved.

One of my best managers in Teck who I worked with, who's a general manager of Red Dog Mine, what he told me was, “Look Preston, at a certain point you have to recognize this is the right thing to do and forget about the value. This is the direction everything's going. Make the leap and do it.”

And so having leadership in place that understands that I think is going to be very important for the growth of this technology in the future. 

Rebecca

Yeah, I mean, I guess, like part of the way that you measure it is with opportunity cost, but it's hard to imagine. You know, say you have people who have previously spent X number of hours preparing a report. If they had this tool, what could they then spend those hours doing that they aren't able to do right now, right?

So it can be difficult to imagine what those other opportunities would be until you've invested and gotten a certain way along your maturity journey. And then you start to see, okay, so now that they aren't spending three hours preparing a report, we can see that they're doing these other activities that are adding value that we wouldn't have foreseen necessarily. 

Preston

Exactly. One of the roles that I held with Teck was lead business improvement for our base metals business unit. And one of the things that I would do and the teams that I would work with would do, when we set up improvements that save different groups time, we'd always ask the question: what are you gonna do with that extra time? What are you gonna work on next? And at that point, it's really important to have a continuous improvement pipeline so that all the different groups  and the functional areas have a roadmap as to what they're going to work on next. And they're going to improve a KPI a certain amount, that's going to save them a certain amount of time or money, then they're going to reinvest that into the next thing on the roadmap. So I think that's how you go about doing that. Great question. 

[music interlude]

AI and community engagement

Rebecca

We haven't talked yet that much about AI. How do you see AI playing a role or, you know, growing into a system like this? 

Preston

Right. So in mining for the past five years, I've seen AI take a greater role in helping to optimize different processes. The first really compelling example that I saw was in milling operations, where the AI tool would show you: this is what your current throughput is, what your production levels are, and then make recommendations on if you did these three top things, it would give you a production uplift of so many percent and it would give you that recommendation based on training from your previous operations data.

But really where I see the next steps are in helping to integrate all the different data systems together very quickly to provide insight. And then beyond that, I think what's going to happen is that we're going to see generative AI used on top of an advanced visualization interface to be able to quickly do the prompt engineering to give you the best answers.

So for example, if you're in an immersive 3D environment and you look at a truck, you don't even know what number the truck is, which asset it is, but you ask to see maintenance history from that viewport, the AI will know this is truck number 56 and it can quickly pull up that information and generate a report for you. 

So that's really where I see it going in terms of advanced visualization in an industrial metaverse. 

Rebecca

I want to ask you about risk, but in two senses. Are there risks that you see with handing this much autonomy potentially over to an AI or using AI more generally in operations for mining? And then the other side of risk is what you think companies who are more hesitant or investing less in pursuing both AI and more advanced visualization, what are they missing out on? 

Preston

Okay, yeah, so very, very good questions. There are a lot of risks across a lot of different areas, but one of the areas that I would be the most concerned about is that the AI is going to enable us to use much less experienced personnel in the roles, who may not be able to determine whether the AI is giving them a good answer or not. And they may just blindly follow the recommendations, not understanding some of the fundamentals of mining, and therefore make bad decisions. I think that's very possible.

And that's just one example and I'll go to the next question, which is around what would mining companies be missing if they didn't adopt this as other mining companies adopt? And I think really it's about the future of work and the incoming workforce.

So we've got an incoming workforce that is video-game native. If you give them a 3D representation of a mine with all this information and you make it intuitive to use, they just understand it and they're able to interact with it. And so you're going to be able to get the very best talent and empower them with the best tools if you go down this path. If not, you're going to end up having a really hard time, which mining companies are already having a really hard time finding the talent and especially the younger workforce.

Rebecca

I think among the general population who don't think a lot about mining, there's a lot of skepticism or wariness around the mining industry because most of the media attention has to do with the ways that a particular mine might have a negative impact on the environment that  surrounds it. But there's on balance, I think, like, less attention generally about how critical these materials are that we're that we're pulling out of the earth. Like they’re, you know, they're the foundation of society in a lot of ways. We wouldn't be able to enjoy the features of an advanced society that we enjoy if we weren't using these materials.

That being said, you know, it is true that a lot of the times our extraction methods, they have a cost, an environmental cost. Do you see an application for—whether it's visualization and/or the combination with AI—and looking for—maybe it's like alternative mining methods or alternative ways to deal with the waste products that come out from processing whatever material it is that you're extracting?

Preston

So I think what will go a really long ways with the community and society in general is being more open with them. One way to do it is to allow them to virtually visit a mine. 

One of the places where we'd seen the greatest uptake of this technology is actually with external engagement. When our people need to go and talk with communities and show them what's going on in the mine and show them future plans for reclamation. 

A picture may be worth a thousand words, but an interactive 3D environment is worth a million. And they can see that it's not this wasteland that they've been told about. There's always active reclamation happening in mines. And they tend to do a very good job nowadays of stewarding the environment.

Mining has a long and not so great history in the past, and really we've changed things a lot to improve and reduce our environmental impact. But people still see the old mining in their mind's eye and so to show them what it actually looks like now and be able to do that very quickly, I think will go a long ways. 

Rebecca

That's interesting. I hadn't thought of using that tool to invite the public, people outside of the company, to come and actually visit. Because you're right, mean, a lot of what the general population sees, if they see anything related to mining, it's like defunct mines or tailings that are left over from many, many decades ago when mining practices were quite different. 

Preston

Exactly. And one other point that I'd like to bring up is, in general, the stereotype about mining is: mining is a dinosaur industry—moves very slow, very averse to change But that is actually changing. 

Mining companies are starting to realize that they need to spend a lot more time and effort in innovation and leveraging these emerging technologies to make their companies and their operations more productive and more profitable. It's happening. It's not happening as quickly as I would like, but when one mining company does, you know, makes a big leap, the other companies are watching. And then they're gonna try to either match that or leapfrog it. I think that's really how we need to get there.

One of the challenges with that is mining companies tend to keep their cards very close to their chest. They'll have innovative things that they'll do but not share it with other miners, fearing that it's gonna give away a competitive advantage.

What I wish for the future was that mining companies were much more open in terms of sharing information and leveraging this emerging technology. And I think that you're going to start to see that much more once we start having these emerging industrial metaverses.

It's going to enable different companies to have sandboxes where they can work together in advancing either products or processes. I really believe that's when—that's the point where mining companies are going to start making the strides that they need to make to really bring the industry into the next era. 

Rebecca

Well, this has been a really interesting conversation, Preston. We really appreciate you coming on the podcast. 

Preston

It's been my pleasure.

Outro

Rebecca

That’s our show for today. 

Joe

You can follow Preston Miller on LinkedIn.

Rebecca

You can find us on all podcast platforms, plus YouTube. And if you like the show, please rate and review us. It helps get the word out. 

Joe

You can also find us on OurIndustrialLife.com. Every two weeks we publish a new batch of stories. Our latest batch includes pieces about emerging battery chemistries, the hydrogen rainbow, and the latest developments in the world of geothermal energy.

Rebecca

We’re also on Substack and LinkedIn and you can email us at our.industrial.life@aveva.com

We’ll see you next time!