Accelerating the
Industrial Internet of Things

Simulation based learning for machines

Improving AI with Simulation technologies

Jose-Maria Lopez 03/13/2017 Technology 46 0


MIT Technology Review has included Reinforcement Learning as a top 10 breakthrough technologies for 2017 in its latest issue.


What is reinforcement learning? It is about to let machines learn by experimenting. As it is explained in the magazine, reinforcement learning copies a very simple principle from nature. The psychologist Edward Thorndike documented it more than 100 years ago. Thorndike placed cats inside boxes from which they could escape only by pressing a lever. After a considerable amount of pacing around and meowing, the animals would eventually step on the lever by chance. After they learned to associate this behavior with the desired outcome, they eventually escaped with increasing speed.

Reinforcement learning algorithms can help, for example, to improve the "driving skills" of self-driving cars. Today’s driverless vehicles often falter in complex situations that involve interacting with human drivers, such as traffic circles or four-way stops. AI engineers can collaborate with simulation engineers to integrate a digital model of the driverless car in a simulated environment, replicating the most complex traffic situations without any risk for people and real traffic. In the virtual space, the control software can perform the maneuvers over and over altering its instructions a little in each attempt. Applying deep learning techniques, the system can extract the patterns from the best performed maneuvers, learning from experimentation.

We see then that high fidelity simulation is going to be a key lever to reinforcement learning but, what kind of simulation we need to do an effective learning? In a former article, I have already discussed about the many applications that simulation brings to the development of smart and connected devices, including the virtual experimentation and design of IoT's products. In this article, I was also discussing about the kind of simulation that it is needed to integrate in an effective way digital models of the new products in simulated environments: we need to work with Net-Centric and interoperable models and simulations. In the same way that the physical product evolves to the Internet of the Things, its digital model will need to evolve to an Internet of the Simulations or IoS, in which heterogeneous simulations of the diverse physical and cyber subsystems of the smart product can interoperate without restrictions.

But to evolve actual simulation products and solutions to the Internet of Simulations, several challenges needs to be addressed, especially in the technologies and architectures to integrate all kind of simulations and real system in a common virtual space. My company, Simware Solutions, has been investing in new simulation technologies for the IoS and the result is our Simware platform. We are also collaborating with lead research groups as the Distributed Systems and Services Research group of the University of Leeds in the development of new simulation architectures and technologies for the Internet of Simulations. Staff and researchers in DSS group are doing very interesting projects related to the application of the Internet of Simulations to the virtual design and experimentation of new vehicles, included self-driving vehicles.

If you are interested in collaborate with us to improve the integrability of cyber-physical systems in simulated environments, send me an email to or contact with my marketing people at

About Jose-Maria Lopez

With Simware Solutions, I am helping to the simulation & training industry to unleash the real power of distributed simulation. Using an open innovation process, we have developed a very disruptive product to create a new market of seamless connected simulators in the network, full of opportunities; the Internet of Simulations or IoS. IoS will expand the simulation to the cloud, allowing full connectivity of devices in the network without any restrictions.

About Simware Solutions

Simware Solutions is leading the introduction of Open platforms into the Simulation & Training markets. Our platform Simware leverages the new Layered Simulation Architecture or LSA to fulfill the requirements of the lead users of the industry, which are demanding open architectures, better interoperability and increasing economical returns for their investments in simulation and training solutions.

Our platform is the first commercial product in the market supporting the Internet of Simulation concept. IoS is about to embrace technologies as internet, distributed systems, open platforms, cloud computing and service oriented architectures for the development and deployment of open, net-centric and interoperable simulations

Simware is the only simulation platform in the market supporting Net-Centric simulation without restrictions, enabling new business models for simulation as the use of simulation as a Service (MSaaS) or the use of simulation platforms as a service (SPaaS).

Our whole portfolio is only based on open standards and APIs, enabling the seamless connectivity and interoperability with almost any other product and solution in the market.


Feature New Record