Project introduction
In 2017, we founded DORPS s.r.o. with the vision of providing meaningful solutions in the field of digitalization using innovative technologies. We were looking for an innovative challenge that would bring together software engineering, systems integration and custom software development based on the client's needs.
We started to build our know-how in the field of Mixed Reality (MR), which offers a fusion of the virtual world with the real one. We saw great potential for the implementation and use of this technology in various fields such as education, healthcare or manufacturing.
As part of the study, we were looking for a suitable device that could be used to create a mixed reality environment. Based on the results of the study, Microsoft Hololens 1 holographic glasses have been chosen. These holographic glasses included a depth camera, which made it possible to map the physical space more accurately and therefore manipulate virtual objects in it better.
With our partner S-Innovations, a systems integration company, we developed a discussion of creating a hardware and software solution for their final client, a manufacturing company focused on production for various types of industries.
Requirements and needs
During the solution development process, we also focused on the following project needs:
compatibility with CAD tools (SolidWorks and AutoCAD) for creating virtual models
selection of suitable SW tools
integration to other systems
accuracy of measurement and mapping of the physical environment
optimization of 3D models and performance
comprehensiveness - to cover areas from visualization, through training to configurations
cost of the solution - achieving an economic return on investment within 3 to 5 years
Solution description
By the way of the detailed analysis, we specified the advantages of using mixed reality in the manufacturing sector. Then we started to
implement a complex solution design for this sector based on the requirements of the final client, respecting the dificulties that mixed reality can bring with it.
As a result of the development, 4 mixed reality applications and a suite of tools that supports these mixed reality solutions were created.
Creating a visualization
The foundation of the entire complex solution was an application for creating visualizations and a set of tools, which were then used and gradually expanded during the development of other applications.
The initiative
Industrial companies confront a wide range of challenges when moving to automated production systems. Most of them appear in the installation of new complex production lines, where unexpected problems are often solved "on the fly" in the validation phase of the operation or through the expensive and time-consuming usage of real-scale models.
The final client was solving serious problems with a new production line and its placement in an existing production hall. In addition to the loss of time, reworking the existing modules of the new production line means a huge increase in costs and thereby a significant delay in the return on the invested funds.
With the use of a 3D visualization application it is possible to avoid these problems and save costs and time. The application is suitable both for the creation of new projects and for the correction of existing, inadequate solutions. Accurate dimensions of real machines with respect to spatial orientation, distances between machines and optimum space for machine operators provides an output for verification and approval by the investor. After thorough checking and approval by the investor or the client, the entire production process (of machines and equipment, e.g. of a new assembly/production line) is forwarded for implementation. Then, the machines and equipment are moved and installed directly into the real environment without any financial losses due to mistakes.
Application functionalities
The application allows visualization of 3D virtual objects and their localization in the user's real physical environment. It is designed primarily for use in manufacturing, commerce and service industries.
With the support of holographic glasses, which the user uses while working, the real space in which the user moves is scanned. The user places the selected object from a catalogue of 3D models in his real space. When gravity is applied, the model falls either to the floor or to another physical or virtual object. The user can further use the set of tools that for example include:
object manipulation such
as moving, rotating and
scaling
distance measurement,
whether between two
virtual objects, two physical
objects, or a combination of
these objects
collision detection between
virtual objects, but also
between physical and
virtual objects
creation of an octagonal
floor plan of the scanned
space together with the
placed 3D virtual objects
Constructing bigger objects
Starting from visualization composition, the focus has turned to the development of applications and tools related to the field of constructing larger objects in space.
The initiative
Each type of production places specific demands on machinery, mobility, supply, room sizes, lighting, sanitation, HVAC, handling areas, warehouses, etc. A huge advantage of the tool we have developed is the ability to easily involve specialists, the future users of the premises, in the design work. With the holographic glasses on, they can "walk around" their future workspaces and evaluate them in terms of suitability for the planned work tasks, ergonomics, safety, etc. They can warn about possible risks or "weak points" of the project. As a result, the building will be perfectly designed without the need for additional and expensive modifications.
Application functionalities
With Holographic Glasses you can start building on the grass green, from the ground floor to the top floor. The projection of an unlimited number of architectural elements will allow their perfect placement in real space. The technology supports the creation of building projects from the very beginning, be it hall objects, their internal parts such as walls, doors, windows, partitions, but also optimizes the process of realizing smaller projects, such as extensions or remodeling of existing buildings, warehouses, etc.
Staff training
We have moved on from the design of workplaces to another issue related to the proper use and operation of workplaces, i.e. training of workers.
The initiative
V čase pretrvávajúceho nedostatku kvalifikovaných pracovných síl sa žiadna firma nezaobíde bez vlastného systému trénovania nových zamestnancov. Pomocou učebnej metódy „learning by doing” cez vizuálne vnemy si človek najrýchlejšie osvojí nové poznatky a vryjú sa mu hlbšie do pamäti, čo skracuje čas potrebný pre zaškoľovanie zamestnancov. Od kvality zaškolenia závisí pracovný výkon budúceho pracovníka.
V prípade, že nie je možné zaúčanie pracovníkov na reálnych strojoch, či už z kapacitných dôvodov alebo kvôli riziku poškodenia daného zariadenia, je možné túto učebnú metódu využiť aj prostredníctvom mixovanej reality.
Application functionalities
The app integrates two of the most important elements of the learning process - a comprehensive visual projection and the personal approach of a professional trainer. Thanks to holographic glasses, the trainee sees realistic models of machines, equipment or products in a real environment.
The trainer prepares the training material in the administration part of the application, where he sets up the training process in the form of steps. The trainer can enter various additional information, animations or annotations. At the same time, he defines the final test to be taken by the employee at the end of the training.
The application in the holographic glasses shows the training process, the steps, models and actions to be completed by the employee in the training.
Service and maintenance
The last part of the proposed solution is an extension of the previous training application concerning the maintenance area in real operation. In this sphere, DORPS has created an application that effectively simplifies and reduces repair and maintenance time.
The initiative
Every downtime on a production line means huge losses proportionally increasing with the length of downtime. Quick service support is crucial in such a case. With the use of holographic glasses, it is possible to easily guide the service technician carrying out repair or maintenance of equipment on the production line.
Thanks to advanced sensors and data transmission, the principles of "remote maintenance" are already being used today in a number of highly robotic manufacturing sectors. According to ABB's analysis, for example, up to 60% of robot errors can be solved by remote way. By combining mixed reality with expert knowledge, there is no need for on-site technicians to spend several days studying an extensive manual - a specialist can guide them visually because they can "see" what they are doing while having a perfect understanding of how the entire production system works.
Application functionalities
The technician can prepare for scheduled maintenance or repair in advance. The instructions also include a list of tools and parts that are needed for the implementation. The technician can still go through the entire step-by-step instructions with the individual tasks he will have to perform in his workplace in his holographic glasses. He can also run through the instructions in the holographic goggles at the place where the repair is to be carried out.
If necessary, he can make use of remote maintenance, i.e. he can call a specialist via the holographic glasses and share the current situation with him. In this case, the remote specialist, who can see the whole situation, instructs him in the repair.
Evaluation and benefits of the project
In designing and implementing such a complex solution, we ensured that the tools were separable and easily reusable. This formed the basis for the mixed reality software toolkit we now call DORPS DEX.
Experience shows that solutions based on combining real images with 3D virtual twins of real objects provide undeniable advantages.
Technological advances in manufacturing and industry go along with innovative solutions. These bring significant benefits in the form of optimization of financial, time and production costs. Our primary goal is to design and develop solutions using these technologies and to adapt them to any need of our clients.