A virtual reality simulator is a combination of computer software and physical devices designed to replicate the response of physical objects based on a user's interactions, with the output displayed through a visually immersive experience and experiencing the movement on the motion base. Various interpretations of this definition exist, but the core essence of a VR simulator lies in its simulation capabilities. Hence, it should adhere to the principles of simulation.
Simulation is defined as “the process of creating a model of a real or theoretical physical system, executing the model on a digital computer, and analyzing the execution output”. Nevertheless, traditional computer simulation lacks real-time user interaction to modify model variables and visualize the results in an interactive manner. Refining the previous definition, a VR simulator can be described as a system of computer programs and interfacing devices that allow users to interact in real-time, calculating physical object responses based on their representations, and presenting the outcomes through three-dimensional graphical objects. It's worth noting that VR simulators may incorporate advanced sensory interfaces such as audio or haptic devices to enhance human perception.
Since their inception in various industries like military, aerospace, automotive, and naval, virtual reality (VR) simulators have served not only as tools for cost reduction in product design and training but also for real-time decision-making. They have become valuable assets for device operation, manufacturing, and process evaluation. Their popularity stems from their capability to provide users with high-quality graphical 3D information on machine operations in an intuitive and real-time manner and with motion base providing users with a heightened sense of immersion and realism by incorporating physical movements and sensations. Another significant advantage is the representation of not just the physical machine but also its interaction with the environment in mathematical or computational form.
1
1.1 VR Simulator Architecture and PipelineWhile a definitive consensus is yet to be reached regarding the essential components and interactions within a VR simulator system, various functional models and pipelines have been proposed as potential prototypes for further development. These key elements are summarized in the diagram below:
The architecture of a VR simulator primarily comprises four main components: physics-based elements, graphical elements, motion base and interactive user interfaces. The physics-based components encompass a simulation engine along with two physics-based models, which represent the machine's physics and the surrounding environment. Meanwhile, the graphical components consist of sub-parts, including graphical machine models and graphical environment models, whereas motion base provides with realistic sensation, all integrated with a built-in rendering engine.
1.1.1 Advantages of VR Training Simulation• Immersive and Engaging Learning Experiences
VR-based simulations strive to replicate reality closely, stimulating learners' intellect and engaging them emotionally. By combining characters, stories, scenarios, sounds, and visuals, VR simulators create a rich learning environment that captivates learners' attention. The interactive nature of VR allows learners to move and interact within the virtual environment, enhancing engagement and fostering effective learning experiences. This heightened sense of presence and interactivity leads to better knowledge retention and promotes critical thinking skills, preparing learners to apply their knowledge confidently in real-world situations.
• Safe and Risk-Free Training Environment
VR training simulations offer a secure space to practice tasks that involve excessive risk, such as electrical wiring repair or dealing with hazardous waste. In a virtual setting, trainees can interact with virtual scenes and hazards without exposing themselves to actual dangers. This capability makes VR simulator training an ideal tool for training personnel in high-risk, potentially dangerous situations without putting them in harm's way. The risk-free environment empowers learners to make mistakes, learn from them, and build their expertise through repeated practice, ultimately enhancing their competence and confidence in handling real-life challenges.
• Highly Effective Skills
VR simulators have proven to be highly effective for up skilling and developing unique skills. Research findings indicate that VR beginners complete their training significantly faster than classroom learners and e-learners alike. Moreover, individuals who use VR simulators exhibit a remarkable increase in confidence when applying their newly acquired knowledge after training. The immersive nature of VR fosters a profound emotional connection and engagement among learners, making VR novices notably more emotionally linked to the content compared to their classroom counterparts. Apart from these research-backed advantages, VR training simulations provide a realistic "real-world" environment, enabling employees to practice and implement concepts, even in hazardous and complex situations. The ability to monitor data and track progress keeps personnel motivated and focused on continuous improvement. Overall, VR training simulators serve as a powerful and efficient tool for enhancing skills and knowledge.
1.2 OverviewWe proudly introduce our state-of-the-art VR-based high-fidelity excavator training simulator. Through accurate integration of driving and excavator controls (joy-sticks) with simulation software, a VR headset, and a motion base, we offer a truly authentic and cost-effective training solution for this extensively used construction machinery. The simulator encompasses diverse training scenarios, simulating various environments, weather conditions, and day-night situations, providing trainees with a comprehensive and adaptable learning experience.
Key Features:
· Realistic and Immersive Experience
· Accurate Excavator Controls
· Diverse Training Scenarios
Benefits:
· High-Fidelity Training
· Safe and Cost-Effective Training
· Skill Refinement and Confidence Building
The construction industry poses significant challenges for workers, especially when it comes to operating heavy equipment like excavators. Workers must learn to control these machines manually, which can be a major and complex task. Excavators play a crucial role in various industrial operations, including excavation, material handling, and grading. However, mastering the operation of excavators requires operators to manipulate a multitude of controls (joysticks, pedals, and switches) in a non-intuitive manner, leading to a lengthy and costly training process.
Existing training methods for excavator operators often lack the realism and practicality needed to effectively prepare them for real-world scenarios. As a result, there is a growing demand for a more efficient and immersive training solution that bridges the gap between theory and practical application.
To address these challenges and meet the specialized needs of construction workers globally, we have undertaken the development of a VR-based high-fidelity excavator training simulator. This simulator aims to provide trainees with a realistic and authentic experience by accurately integrating driving and excavator controls with simulation software, a VR headset, and a motion base. By creating diverse training scenarios in various environments, weather conditions, and day-night situations, we aim to offer a comprehensive and adaptable learning environment for mastering excavator operation.
The primary objective of this project is to revolutionize excavator training, significantly reducing the training period and associated costs, while enhancing skill development and operator safety. The virtual reality environment of the simulator allows users to gain hands-on experience in a risk-free setting, promoting confidence and efficiency in handling these heavy machines.
Through this project, we aspire to contribute to the construction industry's workforce by equipping operators worldwide with advanced training tools that foster expertise, productivity, and safety in excavator operations.
1.3.1 Lack of Operators' PerformanceOperators in the construction industry spend a significant portion of their workday operating heavy machinery like excavators, with the primary goal of accomplishing tasks efficiently and safely. However, past accident reports show that the human factor has a significant impact on accidents. A recent study on earthmoving equipment revealed that operators misinterpreted hazards in more than 46% of cases. Furthermore, analysis of wheel loader accidents shows that accidents involving mining and non-mining workers being hit, run over, or crushed by wheel loaders account for almost 41% of all wheel loader-related fatalities. These statistics emphasize the urgent need to enhance operator awareness and skills to ensure the safety and well-being of operators, other personnel, and the surrounding environment. Improving operators' performance can play a crucial role in mitigating accidents and improving overall operational safety in the construction industry.
1.4 Specification of VR-based High Fidelity Excavator Training simulatorThe VR-Based High-Fidelity Excavator Training Simulator project offers a cost-effective and efficient solution for training individuals in heavy construction machinery operation. Leveraging VR-based training simulators, construction companies can provide comprehensive training without the need for physical training sites or risking any damage to expensive machinery. The simulator ensures a safe and realistic learning experience within a virtual environment.
The project is divided into two main parts: software and hardware.
Software Specifications:
Unity 3D: The simulator is developed using the Unity 3D software, allowing the creation of immersive 3D environments with backgrounds, terrains, meshes, objects, and models.
Blender: Utilizing blender enhances the creation and customization of detailed 3D models, adding to the realism of the simulation.
Logitech SDK: The Logitech software development kit facilitates seamless integration of Logitech hardware components with the simulation.
Hardware Specifications:
Logitech 3D Joysticks: The simulator uses Logitech 3D joysticks to replicate excavator controls accurately, ensuring a realistic and intuitive learning experience.
Logitech Steering Wheel: The integration of a Logitech steering wheel enhances the simulation of vehicle operation, providing a comprehensive training platform.
Logitech Pedal Set: The Logitech pedal set enables trainees to control acceleration and braking, further enhancing the authenticity of the training experience.
Motion Base: The motion base adds an unparalleled level of realism to the simulator, replicating the movements and vibrations of the excavator, giving trainees a true-to-life feel while operating the machinery.
The seamless integration of software and hardware components allows for a cohesive and effective VR-based training simulator. Trainees can practice and refine their excavator operating skills within a risk-free virtual environment, boosting operator confidence and safety.
Overall, the VR-Based High-Fidelity Excavator Training Simulator serves as a cutting-edge tool, revolutionizing excavator training and ensuring well-trained operators worldwide.
1.5 Aims and ObjectivesAim:
The primary aim of this project is to address the safety hazards and operational challenges associated with onsite excavator operations in the construction and other industries. The project seeks to provide a cost-effective and efficient solution through the development of a VR-based high-fidelity excavator training simulator.
Objectives:
Safety Enhancement: Develop a realistic and immersive VR training simulator that allows operators to practice excavator operations in a safe virtual environment. By offering hands-on training without real-world risks, the simulator aims to minimize accidents and safety hazards.
Cost Reduction: By simulating excavator operations, the project aims to reduce fuel consumption, maintenance costs, and repair problems associated with real machinery training. Providing a cost-effective training alternative, the simulator will help industries optimize their resources.
Quick and Efficient Training: The simulator intends to enable new workers to learn excavator operations quickly and efficiently. By providing a realistic training experience, trainees can gain practical skills that can be directly applied to real-world scenarios.
Educational Collaboration: The project aims to facilitate Construction Technology and Training Institute (CTTI) by providing them access to the simulator. CTTI can utilize the simulator to enhance the training of their students and workers, equipping them with valuable hands-on experience.
The VR-Based High-Fidelity Excavator Training Simulator project strives to revolutionize excavator training by ensuring operator safety, reducing operational costs, and offering efficient skill development. By collaborating with CTTI, the project aims to extend its benefits to the construction industry and contribute to the overall growth and safety of the workforce.
1.6 ApplicationThe VR-Based High-Fidelity Excavator Training Simulator holds significant potential for a wide range of applications, playing a crucial role in enhancing safety, efficiency, and skill development across various industries.
Construction Industry Integration: The simulator will be introduced to different industrial companies and construction industries, revolutionizing their approach to excavator training. By incorporating the simulator into their training programs, these industries can eliminate hazardous on-site practices, ensuring workers can operate in a safer environment.
Cost and Resource Savings: The adoption of the simulator results in substantial cost savings for both workers and companies. Reduced fuel consumption, minimized maintenance and repair costs, and decreased downtime translate into improved operational efficiency and economic benefits for the industries.
Training and Skill Enhancement: Students and workers in construction-related fields can utilize the simulator to practice operating excavators and gain better hands-on experience. By offering a realistic training platform, the simulator empowers trainees to refine their skills, resulting in a competent and skilled workforce.
CTTI: The Construction Technology and Training Institute (CTTI) will be the inaugural site for the simulator. By integrating the simulator into their training curriculum, CTTI can equip their students and workers with cutting-edge training, preparing them for real-world challenges in the construction industry.
Overall, the application of the VR-Based High-Fidelity Excavator Training Simulator extends to diverse sectors, fostering safer work environments, cost-effectiveness, and skill enhancement. By promoting efficient and realistic training, the simulator contributes to the growth and success of the workforce while elevating industry standards in excavator operations.
1.7 Strategy1.7.2 Capstone Phase IIThis report is organized into six chapters.
• Chapter 1 describes a typical VR simulator system in terms of its important components and the parts of these components and the project introduction and specification with the project planning.
• The Literature review is proposed in Chapter 2 to provide basic information on the previous excavator training simulator research and used technologies.
• Project design is described in Chapter 3 to provide all development procedures, methodology, and detail about simulation modeling.
• Tools and Techniques are explained in Chapter 4 to provide all information about hardware and software technical specifications.
• All project results are included in Chapter 5 including all hardware and software outcomes as required with comparing to other excavators.
• In Chapter 6 we concluded our project and provides all the references from where we took all our help during the project.
Chapter 2LITERATURE REVIEWLooked through various research papers on the project and got some insights that were very helpful in completing the project.
2.1 Related WorkThere are many Excavators Simulation has been introduced few of them are as follows:
2.1.1 VR Simulator Using Leap MotionA leap motion controller and a digital headset are used as devices in this excavator simulation system. The Digital Truth headset used with this device is the Oculus Rift. Head movements of the person wearing the Oculus Rift are sent to the digital camera. Users can freely turn their heads to see inside the gadget's digital world. In addition to the digital excavator, a digital challenge website is also available. Leap Motion Controller, on the other hand, is used as a device for digitally managing gadgets. Gadgets are controlled by real finger movements. Consumer hand gestures are very important in controlling the simulator. The operator of this machine is his digital joystick and the hand holding the lever. While the digital hand is colliding with the digital joystick or lever, the person must perform the clutch gesture. After the grab gesture is maintained, the person wants to move on. Grab and slide the digital joystick or lever and the excavator will go through the joystick or lever function. Feedback from consumers is necessary to get real results from the developed system. These feedbacks are evaluated as individual decision-making stage and consumer satisfaction.[2]
2.1.2 A 3d Physics-Based Hydraulic ExcavatorReal-time simulation techniques from multibody system dynamics allowed the development of a realistic but computationally efficient 3D physics-based model of a complex machine like a hydraulic excavator. The resulting simulator, which runs in a standard PC, can reproduce almost all the maneuvers performed by real excavators. The project has finished the software development stage, and now is starting the human evaluation stage: the simulator will be tested by experienced operators, and their findings will be used to adjust the mathematical model of the excavator (contact forces and other data not supplied by the manufacturer) to increase the realism of the simulator.[3]
2.2 Limitations and BottlenecksLeap motion controller and Oculus Rift remain difficult for excavator operators to master. New controls that are more precise and easier to learn are needed to develop excavator controls to achieve better results.
2.3 SummaryThe first technology is still in development and is an excavator simulator that can be controlled by a Leap Motion Controller. A real hand appears as his 3D hand model, moving in real-time with the movement of the real hand. These 3D hand models take control of the system by grabbing the virtual joysticks and levers of the virtual excavator. New controls that are more precise and easier to learn are needed to develop excavator controls to achieve better results [2]. The second technology was used to build a pair of PC-based Show Digital Truth devices for excavator simulators. Enrich your scenes with meaningful detail with real-time terrain distortion and GPU-based texture blending. By inserting a photo into a dedicated monitor, the operator has an uninterrupted and immersive digital environment. Drilling force can be used to determine the most useful course of work and reverse mechanics can follow the mannequin [3].
Chapter 3
DESIGN AND IMPLEMENTATIONA hydraulic excavator is a construction dredge primarily powered by a hydraulic system. The excavator's hydraulic system transfers the hydraulic flow generated by the hydraulic pump to the working area through the hydraulic fluid. The external representation refers to some structural parts closely related to the internal hydraulic system. The external representation should include properties such as relative rod position, rod dimensions (length and width), rod stretch, and rod weight. Important assets identified in the internal and external representations are used to calculate the effective physical response of the machine. The motion calculation begins with the calculation of piston displacement given hydraulic flow and piston cross-sectional area, with a motion time segment formulated as follows:
Piston Displacement = (flow rate / ram cross-sectional area) × segmental time duration
Once this calculation is performed for the ram (cylinder) corresponding to the maneuver, the spatial information of the rod can be calculated through a series of geometric analyses. [4]
3.1 Design and DevelopmentThe project is divided into 2 parts; hardware and software. The software part consists of creating a simulator by using Unity 3D software where we created backgrounds, terrains, meshes, objects, and models in the 3D environment. On the other hand, hardware parts include joysticks, steering wheel, pedals, Oculus Rift S VR headset which are interfaced with the simulation software through USB interfaces and Motion Base which is interfaced through Arduino with simulation software through USB interface.
• Firstly, we have tried three different software (i.e., Unigine, Unity 3D, and Unreal Engine) and the best one we have selected is Unity 3D because of the easy availability of its tutorials and its vast assets store.
• Unity3D is a powerful cross-platform 3D engine and easy-to-use development environment. Simple enough for beginners and powerful enough for experts. Unity should be of interest to anyone looking to easily create 3D games and applications for mobile, desktop, web, and consoles. By using the software Unity 3D, we developed the simulator by creating the background/terrain. Using the excavator asset, we did scripting and set the collision for the objects. Added different objects like trees and stones. Then we picked the stones using the arm of the excavator and applied excavator physics there.
• Hardware parts include the Steering wheel, pedal set, joysticks VR headset and motion base. Furthermore we plan to arrange the three projectors and combine them to display. Also, we have thought of using a cabin to give our simulator a proper shape so that it looks like an original excavator.
• We have to interface software with the hardware. Make sure that they must align, and software controls must be linked with the hardware control. Timing is an important part hence there should be no delay so that it works precisely and accurately. For this interference, we changed the scripting and amend the codes, and did scripting again.
• For the motion base to achieve precise alignment between the motion base and the virtual excavator's movements in the simulation, multiple trials were conducted. The main goal was to synchronize the motion base with the excavator's positioning accurately, especially during changes in terrain, such as steep inclines or declines.
• Through careful observation and analysis of the simulated excavator's movements, essential parameters were extracted to control the motion base. The collected data allowed for adjustments to the motion base's control system to ensure its movements closely mirrored the virtual excavator's actions.
• By fine-tuning various parameters and feedback mechanisms, a seamless correlation was achieved between the virtual excavator's maneuvers and the motion base's responses. The ultimate aim was to enhance the overall realism and immersion of the simulation, providing users with a more authentic experience for training, testing, and other simulation purposes.
• To seamlessly integrate the Oculus Rift S VR headset with the simulation, several technical considerations come into play. One crucial aspect involves meticulously calibrating the floor level within the Unity 3D environment to ensure an optimal virtual reality experience.
• Precise calibration of the floor level is essential to ensure that the user's virtual movements align seamlessly with their real-world physical movements, the user's real-world floor level is established as a reference point. This reference point serves as the foundation for positioning the virtual ground plane within the Unity 3D environment.
• In the end, we have to compile everything including software and hardware so that if we want to start the project the hardware and software both should start at the same time. Our Simulator is ready to work it can pick objects like stones from the ground and place is in different places and get to know how the excavator works.
3.3 Hardware Specifications3.3.1 Steering WheelIt is from the company Logitech. It is used to control the directions and movements of the excavator towards the horizontal X axis.
It is from the company Logitech. It is used to control the race and reverse of the excavator towards the vertical Y axis.
The Extreme 3D Pro Joystick from Logitech. It features a handgrip-designed twist grip, 12 programmable buttons, an 8-way hat switch, a rapid-fire trigger, and a weighted base. Once set up, Logitech's gaming software can be used to configure button assignments and macros to assist.
• Twist Rudder Control
• 12 Buttons
• 8-Way Switch
• Rapid Trigger
• Stable, Weighted Base
It is used to control the movements of the excavator in every direction. Also, it helps the excavator to move the bucket, stick, boom, and swing.
The VR Headset Rift S from Oculus. Designed to deliver an immersive and realistic virtual experience. Its specification features Crystal-Clear Visuals, wide field view, precise tracking, and spatial audio.
Display:
· Display Type: Fast-switch LCD
· Resolution: 2560 x 1440 pixels (1280 x 1440 per eye)
· Refresh Rate: 80 Hz
Field of View:
· 155 degrees
Tracking:
· Inside-out Tracking System
· 6 degrees of freedom (6DoF) for head and hand tracking
· Oculus Insight technology
Audio:
· Built-in stereo speakers
· 3.5mm audio jack for external headphones
Weight:
· Approximately 563 grams
Dimensions:
· Headset (folded): 10.94 x 6.30 x 7.87 inches (H x W x D
3.3.5 Motion BaseThe motion base is a critical component of our simulator, providing a dynamic and immersive experience for users. Our team undertook a meticulous approach in its creation, considering various dimensions and engineering aspects to ensure optimal performance. The motion base frame, carefully crafted from durable metal, forms the robust foundation of this system.
Hardware Components:
· LBT2 Motor Driver:The LBT2 motor drivers, renowned for their precision and reliability, play a key role in controlling the motion actuators of our simulator. These drivers facilitate seamless communication between the software and hardware, enabling precise and synchronized movements.
· Windshield Wiper Motors: Selected for their power and efficiency, the windshield wiper motors serve as actuators to generate the motion effects. Our thorough testing and assessment confirmed their suitability for the demanding requirements of the simulator, ensuring smooth and realistic motions.
· 12v 20 Amp DC Power Supply: To deliver consistent and reliable power to the motion base, a robust 12V 20 Amp DC power supply was integrated. This power supply unit meets the high-power demands of the motors and ensures the stability of the entire system during operation.
· Arduino:The Arduino microcontroller acts as the brain of the motion base, orchestrating the precise control and synchronization of the various components. Its versatility and programmability empower us to tailor the motion profiles according to the specific scenarios encountered during the excavator training simulations.
3.4 Software Specifications3.4.1 Design of Excavator ArmAs excavator is heavy machinery it has different parts including bucket, stick, boom, and swing. The bucket is the part in which the material is picked up or dig from the ground. It has sharp ends that help in picking. Then comes stick this is directly attached to the bucket and help the bucket to move more easily covering more angle and distance. Similarly, boom is used to help the arm excavator to move and cover more angle and distance to make excavator more useable. The last part is the swing which help the excavator to move as it help the whole body of excavator to move the body is quite heavy hence it take time to move. In our excavator lower portion is attached to wheels on which the heavy parts are attached.
F = 10,350N
r = 392mm
L = 316mm
P = 15.7N/mm2
A = 66838mm2
Shear stress = 41N/mm2
Bending stress = 85N/mm2
Let, d = diameter of pin
Torque, T = F × r = 10300×392 = 4027400N-mm,
Load, W = P×A = 15.7×66838 = 1048656.9N
Now,
Max. bending moment = M = W*L/4 = (1048656.9 × 316)/4 = 82581724Nmm
Since, the pin is subjected to a suddenly applied load.
Considering,
Km = 1.5
Kt = 2
We know that, equivalent twisting moment Te = (Kt × T)2 + (Km × M)2
= (2 × 4027300)2 + (1.5 × 82581724)2
Te = 124134176.5 N-mm
But,
Te = π/16 × d3 × shear stress 124134176.5 = π/16 × d3 × 42
Therefore,
d = 246.9mm
we know that, an equivalent bending moment
Me = 1/2[Km × M + Te] = ½[ 1.5 × 82581723 + 124134176.5]
Me = 124003380.5N-mm
But,
Me = π/32 × d3 × bending stress 124003380.5 = π/32 × d3 × 84
Therefore, d = 246.86mm
Taking the larger of the two values,
we have the diameter of pin = d = 246.9mm3.4.1.2 Calculation for the total amount of material the bucket can liftSince,
we have considered a light duty construction work. So, calculate a soil surface.
Density of soil is 1463kg/m3
For the existing model,
Volume of bucket V= 0.022m3
Total weight of soil,
W= Density × Volume = 1463 × 0.022 = 32.186 Kg
The calculated volume the of bucket,
V= 0.028m3
Total weight of soil, W= Density × Volume = 1463 × 0.028 = 40.964 Kg
Self-weight of bucket = 17 Kg
Now,
Total load acting on bucket = Self weight of bucket + Total weight of soil
=17+40.964 = 57.964Kg = 57.964×9.81=568.62N
By using this load, the dynamic analysis will be done [4].
3.4.3 Design of SimulatorThis session explains the Unity interface, menu items, the usage of Assets, and developing Scenes:
• Downloading, installing, and activating Unity, the usage of the Hub, and managing to control your licenses.
• A “package” is a container that holds our all types of features or assets such as:
§ Editor tools and libraries
§ Runtime tools and libraries
§ Asset collections
§ Project templates to share common project
• Excavator terrain meshes contain data for working, a statistic that describe a shape.
Unity uses meshes in the following ways:
Graphics share meshes with materials, shapes appearance and shapes of colliders.
• When using meshes in C# scripts, the Mesh category provides scripting access to an object's mesh geometry like for steering wheel the code given bellow.
• Textures are just generic bitmap pictures used on mesh surfaces. we can assume it's a photograph of a texture, printed on a sheet of rubber that is stretched and fixed to the mesh as shown in Fig. 3.12 samples are given which we used on our terrain and assets.
• In Excavator Simulation, inflexible bodies enable physics-based behavior such as motion, gravity, and collisions which are implemented on the all the rigid bodies as shown in Fig. 3.13 and Colliders are used to set up a collision between all these GameObjects to combine them all.
Motion simulators are specialized devices or systems designed to recreate the sense of movement and motion experienced in real-world scenarios or virtual environments. They play a crucial role in various industries, including aviation, automotive, entertainment, and training, by providing users with an immersive and realistic experience. These simulators are equipped with mechanisms that allow them to move along different axes, known as degrees of freedom (DOF), which define their capabilities and the complexity of motion they can generate.
3.5.2 Degrees of Freedom (DOF):Degrees of freedom refer to the number of independent axes along which the simulator can move. A higher number of DOF enables the simulator to simulate more complex and realistic motions, enhancing the user experience. Here are the common types of motion simulators based on DOF:
- 1-DOF Motion Simulator: A 1-DOF motion simulator provides motion along a single axis. It typically allows back and forth or up and down movements, making it the simplest form of a motion simulator. These simulators are commonly found in basic amusement park rides, where the motion is limited to a single direction.
- 2-DOF Motion Simulator: A 2-DOF motion simulator incorporates two independent axes of motion. This type of simulator can offer pitch (forward and backward) and roll (side-to-side) movements. It is commonly used in arcade games and some flight simulators, providing users with a more engaging experience compared to 1-DOF systems.
- 3-DOF Motion Simulator: A 3-DOF motion simulator further enhances the user experience by adding an additional axis of movement. In addition to pitch and roll, it also allows for yaw (rotation around a vertical axis). These simulators are widely used in advanced flight simulators, driving simulators, and virtual reality setups, offering users a more realistic sensation of movement.
- 6-DOF Motion Simulator: A 6-DOF motion simulator represents the most sophisticated type commonly used in professional flight training and aerospace research. With six independent axes of movement, including pitch, roll, yaw, surge (forward and backward), sway (side-to-side), and heave (up and down), these simulators provide a highly immersive and authentic experience.
- 9-DOF Motion Simulator: Some specialized applications require even more freedom of movement. A 9-DOF motion simulator goes beyond the six-axis system by incorporating rotational movement along the three perpendicular axes (X, Y, Z). This level of complexity is typically used in advanced research and development environments.
The use of motion simulators is widespread in various industries to provide users with realistic and immersive experiences of movement and sensation.
The degree of freedom used in a motion simulator is an important factor that determines how realistic and complex the simulated motions are. Why?
A variety of simulators, ranging from basic 1-DOF systems to complex 6-DOAF and 9-DO F systems, are available for various applications and training requirements .The motion simulator chosen will be influenced by the project or application's specific requirements, budget, and desired degree of realism.
To demonstrate our ingenuity and creativity, we created a 2-Dof Budget Simulator to demonstrate the ability of our project team to deliver exceptional results within their limited budget.
Our team took on the challenge and created a cost-effective yet highly sophisticated solution that not only meets project specifications but also demonstrates our commitment to excellence in all aspects. The 2-Dof Budget Simulator was created with innovative ideas in mind, taking into account the value and efficiency of each component before attempting to create a product that could compete directly with its production. We worked in tandem to develop the most efficient version, leveraging existing technologies and methodologies while also preserving the best possible quality of the product .Our design process was centered on making the simulator user-friendly and intuitive, while also providing seamless navigation and interaction. We adhered to best practices in the industry to ensure reliability and accuracy, providing stakeholders with accurate budget estimates and insightful analysis.
The budgetary challenges were undoubtedly challenging, but they also motivated us to explore new opportunities.
3.5.2 Hardware Components and its Working:Below is an explanation of the components used in our 2-DOF (Degree of Freedom) simulator:
1.Metal Frame:
The solid metal frame is the foundation of our 2-DOF simulator.
By providing a strong foundation, it ensures that the components are supported and remains stable enough to allow users to participate in simulation activities without harm.
With careful consideration to its geometry, the frame provides a level of stability that is
necessary for achieving believable and immersive simulation experiences.
2.U-Joint:
A Universal Joint, also known as Universal Joint, is utilized to transfer motion from one rotating shaft to another at different angles.
Our simulator's allows for precise articulation of the two degrees of freedom, making it possible to replicate real-world movements. This is particularly useful in this context.
By using this versatile component, the simulator can mimic intricate movements with precision, resulting in an authentic and engaging simulation experience.
3. WindShield Wiper Motor:
The Windshield Wiper Motor is a critical component of our 2-DOF simulator, acting as an actuator to generate the required motion for various dynamic scenarios.
With its strong and stable performance, it is the perfect fit for providing smooth and lifelike movements.
The motor's controlled rotation, along with our advanced programming, creates a sense of immersion in the simulated environment, which enhances the simulation realism.
4. IBT-2 Motor Driver:
The integration of an IBT-2 Motor Driver as an interface between the Arduino controller and the Windshield Wiper Motor marks a significant advancement in our 2-DOF simulator's capabilities. This dedicated component serves as a crucial intermediary, empowering us to finely control the motor's speed and direction with unparalleled accuracy.
With the IBT-2 Motor Driver in place, we gain precise command over the Windshield Wiper Motor, allowing us to orchestrate a diverse array of simulator motions. Users can now experience a wide range of sensations, from gentle turns to more dynamic and thrilling movements, all intricately synchronized with the simulation's environment.
The IBT-2 Motor Driver's ability to manage speed and direction ensures that each motion is executed smoothly and seamlessly, enhancing the overall realism of the simulation. Its responsive control enables us to create immersive scenarios that closely mirror real-life experiences, making the simulation more captivating and engaging for users.
Additionally, the inclusion of this specialized driver contributes to the simulator's safety and reliability. With such precise control, we can carefully regulate motion parameters and prevent any sudden, jarring movements that might compromise user comfort or safety.
In summary, the integration of the IBT-2 Motor Driver as an interface between the Arduino controller and the Windshield Wiper Motor elevates our 2-DOF simulator to new heights. By harnessing the driver's precision control capabilities, we can offer users an enhanced range of sensations, ensuring that their simulation experience feels authentic, exhilarating, and safe. This advancement represents a significant stride forward in delivering a truly immersive and captivating simulation environment.
5. 12V 20 amp DC Power Supply:
The motor of the simulator requires a consistent power supply that can be powered.
A 12V 20Amp DC Power Supply is the preferred choice for powering the Windshield Wiper Motor and other electronic components. This device requires no additional equipment.
With its robust performance in both high-pressure environments and large current capacity, it can handle the demanding simulation workloads that come with our simulator.
6. Arduino:
Our 2-DOF simulator's central component is the Arduino microcontroller, which functions as the brain by analyzing data and carrying out control algorithms.
By utilizing this versatile and programmable board, we can incorporate user inputs into the simulation, sensor feedback, and motion calculations to control the simulator's movements.
Our users can benefit from our user-friendly simulation models, which are crafted using the Arduino code and have unique features.
3.5 SummaryThis chapter contains all the mediatory initial steps and information needed to work on the project. This information related to software and its working is explained also, hardware modules their connectivity steps are explained in detail that how can we connect this hardware with each other and the PC with the inbuild software’s Logitech SDKs.
Chapter 4
TOOLS AND TECHNIQUES4.1 Technical Specifications of HardwareFollowing hardware had been used for developing VR Based Excavator Training Simulator:
• Logitech 3D joysticks
• Logitech Steering Wheel
• Logitech pedal set
4.1.1 Logitech Steering WheelWe are controlling the steering wheel in the horizontal direction. Which only can turn left and right.
4.1.2 Logitech Pedal SetThe Pedal Set consists of three pedals that control the vertical movements of the excavators. From the leftmost pedal, we are controlling the speed/ race of the excavator, which helps our excavator to move only in the forward direction. From the middle pedal, we are controlling the brakes or reverse of the excavator, which helps our excavator to move only in the backward direction.
4.1.3 Logitech JoysticksA joystick is an input device that can be used for controlling the movement of the cursor or a printer in a computer device. Joysticks have eight movements we can say it has four degrees of freedom(joints).
4.1.3.1 Right Joystick
If we move the right joystick in the left and right direction it will move the bucket of the excavator in the software. If we move the right joystick in the up and down direction it will move the stick of the excavator in the software.
4.1.3.2 Left Joystick
If we move the left joystick in the up and down direction it will move the boom of the excavator in the software. If we move the left joystick in the left and right direction it will move the swing of the excavator in the software.
4.2 Calibration of Hardware2 calibration software have been used for the testing of controls and for assigning the function of buttons or movements that have to be performed through scripting
4.2.2 Logitech SDKThe Unity 3Dpackage enables control of Logitech Gaming products. The Logitech Gaming SDK is divided into five categories:
• Logitech|G Arx Control App SDK
Easily create a second screen experience for the PC Game through an applet running on Logitech|G Arx Control app.
• Logitech|g LED Backlighting
Access backlighting features of Logitech|G devices, now including also per-key color backlighting on featured devices.
• Logitech|G G-key SDK
Receive notifications on G-keys events coming from Logitech|G featured mice, headsets, and keyboards.
• Logitech|G LCD SDK
• Logitech|G Steering Wheel SDK
Control Logitech Gaming controllers, such as steering wheels, flight controllers or gamepads.
4.3 Software SpecificationsSoftware named “Unity 3D” on this software we developed terrain and made changes accordingly. Then we imported the asset of the excavator and did its coding of movements.
The collision was also done between different objects. Like:
4.3.2 Excavator Controls• Engine
Pressing the Y button on the keyboard enables all the controls by starting the engine of the excavator. When the power button is pressed it starts the engine and we can control these attributes.
• Sounds
When the engine starts, I produce the real excavator sound of the Engine, indicators, start, stop, and hydraulic forces.
• Smoke
We have added the smoke to the excavator. When the engine starts the excavator starts producing smoke also, we can control the particle limits, playback speed, playback time, and speed range.
• Indicators and Headlights
Headlights and Indicators were added which can be controlled through a keyboard with buttons like Q for the left indicator and E for the right indicator
4.3.3.1 Excavators Arm
The excavator’s arm controls the four Degree of Freedom (DOF) means four joints or 8 movements:
4 DOF
8 Movement
Boom
Boom Up
Boom Down
Swing
Swing Left
Swing Right
Stick
Stick Up
Stick Down
Bucket
Bucket Open
Bucket Close
4.3.3.2 Excavation
In this part we wanted our excavator to pick the stones from the ground which can be placed or dropped in any other place by controlling all 4 joints' degrees of freedom.
- Set up Oculus Rift S and Software:
Begin by ensuring that you have the Oculus Rift S headset and controllers properly set up and configured on your PC. This includes installing the Oculus software, which you can download from the official Oculus website.
- Install Oculus Integration for Unity:
Open your Unity project and navigate to the "Window" menu. From there, access the "Package Manager" and click on the "+" button to add a package from a Git URL. Enter the URL for the Oculus Integration package, which is available on GitHub. This will initiate the download and installation process.
- Enable VR Support in Unity:
To enable VR support in Unity, go to "Edit" > "Project Settings" > "Player." Under "XR Settings," make sure to check the "Virtual Reality Supported" option. Then, click on the "+" button and add "Oculus" as a Virtual Reality SDK. Prioritize Oculus at the top of the list to make it the default VR platform for your project.
- Set Up Camera Rig:
In your Unity scene hierarchy, remove the Main Camera if it exists. Instead, use the "OVRCameraRig" prefab from the Oculus Integration package. This prefab contains camera and controller components optimized for Oculus devices.
- Implement VR Interactions:
For interactive elements in your VR experience, you can use the "OVRGrabbable" component. Attach this component to objects you want users to interact with and grab in the virtual environment. For hand tracking and interactions, utilize the "OVRHand" components for both left and right controllers.
- Test in VR
Before proceeding further, ensure your Oculus Rift S is connected to your PC. Make sure your Unity project is configured correctly for VR, with Oculus selected as the active VR SDK. Click on the "Play" button in Unity to test your project in VR mode. Once you wear your Oculus Rift S headset, your Unity project will be displayed in VR, and you can interact with the virtual world.
- Build and Deploy:
Once your VR application is ready for deployment, access "File" > "Build Settings" in Unity. Select the desired platform, such as Windows PC. Click on "Build and Run" to generate the executable file and automatically launch it on your Oculus Rift S headset.
Top of Form
4.3.4 Hardware and Software Connection:To establish a SerialPort between Unity and Arduino.The SerialPort.IO library requires us to establish a serial connection between the two devices. SerialPort.IO library facilitates communication with the serial port on the computer running Unity, which permits data transfer to and from the connected Arduino board.
The procedure for setting up communication is as follows:
Ø Install SerialPort.IO Library
Ø Setting Up the Arduino
The process of setting up the Arduino board involves connecting it to your computer via a USB cable.
Upload a sketch to the Arduino IDE, which can read data from Unity using the serial port and perform necessary actions (e.g.).
4.4 SummaryAll software and hardware implementation explained in this chapter where we have clearly explained the function of all joints, smoke, sound, effects, engine, headlights, and indicators to control them also mentioned the assignment of variables of the controls in the figures.
Chapter 5CONCLUSIONSIn this chapter, we explained all our outcomes and results as our goal to achieve at the end of the project.
5.1 Results and DiscussionsA description of the results of the project with hardware and software working.
5.1.1 Software Results• Excavator Model
The output of combining all the small parts of excavators to build up a whole working excavator with all accurate joints.
• Wheel Colliders
Applied successfully wheel colliders to our vehicle and terrain so that the vehicle can drive on the ground when interacting.
• Indicator and Headlights
Added indicators lights and headlights which are currently controlled by a keyboard to turn on a headlight press ‘h’ and for indicators press ‘q’ for the left indicator and ‘e’ for the right indicator.
• Center of Gravity
Added center of gravity to the vehicle to drive smoothly while driving or picking stone, its code is attached in car control’s part.
• Terrain Model
The colliders also applied on meshed terrain so the vehicle can drive on it and add excavator’s scenes to it
• Excavator Canvas
Currently working on the dashboard where we can see fuel level and arm motions angle for better accuracy.
In hardware, we are used Logitech 3D joysticks Logitech Steering Wheel
Logitech pedal set
Motion Base
Oculus VR Headset
• Steering Wheel
We are controlling the steering wheel in the horizontal direction.
• Pedal Set
From the leftmost pedal, we are controlling the speed/ race of the excavator, which helps our excavator to move only in the forward direction. From the middle pedal, we are controlling the brakes or reverse of the excavator, which helps our excavator to move only in the backward direction.
• Right Joystick
If we move the right joystick in the left and right direction it will move the bucket of the excavator in the software. The bucket will open up to 5 degrees when the joystick is moved towards the left. The bucket will close up to -160 degrees when the joystick is moved towards the right.
Right Joystick Joints
Angles
Stick Up
70
Stick Down
-41
Bucket Open
5
Bucket Close
-160
If we move the right joystick in the up and down direction it will move the stick of the excavator in the software. The stick will lift to 70 degrees when the joystick is moved up.
The stick will down, up to -41 degrees when the joystick is moved down.
• Left Joystick
If we move the left joystick in the up and down direction it will move the boom of the excavator in the software. The boom will lift to 29 degrees when the joystick is moved up. The boom will down, up to -40 degrees when the joystick is moved down.
If we move the left joystick in the left and right direction it will move the swing of the excavator in the software. The swing will move the whole 360 degrees and along this, the upper part of the excavator including its cabin will also move up to 360.
Left Joystick Joints
Angles
Boom Up
29
Boom Down
-40
Swing Left
360
Swing Right
360
• VR Headset
In the VR headset, the left joystick's up and down movements controls the excavator's boom. Leaning the joystick upwards will elevate the boom to a 29-degree, while leaning it downwards will lower the boom to -40 degrees.
Additionally, the left joystick's left and right movement measures the swing of the excavator. This swing offers a complete 360-degree rotation, and as the upper part of the excavator, including its cabin, moves along, it will also have a range of motion up to 360 degrees.
These user-friendly controls allow for an immersive and authentic construction site experience, enabling users to operate the virtual excavator intuitively and realistically as if they were sitting inside the actual cabin.
• Motion Base
With the VR headset and simulation from the software the motion base gives the movement. When the position in the excavator changes it tilts the motion base seat accordingly giving the immersive feel to the operator.
5.2 LimitationsLimitations may include the different controls for different excavators. It may be possible the excavators of the different companies have different controls that are developed in the software for simulation that may affect the working. As this is a wheeled excavator it may perform unevenly on terrain.
5.2.1 Using Leap Motion ControllerThis study uses an excavator simulator under development that can be controlled using the Leap Motion Controller. A real hand appears as his 3D hand model, moving in real-time with the movement of the real hand. These 3D hand models take control of the system by grabbing the virtual joysticks and levers of the virtual excavator [6].
The investigation is required to obtain user acceptance and controller satisfaction results. Survey respondents for this system are 25 excavator operators. After research, no statement has an average score to achieve a value that agrees and satisfies. The average score of the user awareness survey results is 3.44 points for "the camera moves according to the movement of the head". The highest average score in the user satisfaction survey is "I am satisfied with the joystick function that moves the excavator", with an average score of 3.16 points. New controls that are more precise and easier to learn are needed to develop excavator controls to achieve better results [5].
5.3 RecommendationsOne can work and convert limitations into future work which we were not able to do due to a shortage of time.
Also, they can change the terrain and make it according to the background they want because the asset part will be almost the same but if they want to add something else like different types of buckets, etc., they can go for it too.
5.4 SummaryThe main objective of our project was to achieve the working simulation of excavator controlling with actual hardware of any company (JSR company). Explained all the outcomes of software and hardware with one-by-one module screenshots are also attached.
CONCLUSIONIn this Project, a VR based excavator training simulator has been developed is presented for developing a new type of construction heavy equipment training Earthmoving site Trainee-controlled Excavator simulators based on the integration of the actual site data and a multi-agent system in an immersive VR environment. The presented Working model and discussed project are good indications of the feasibility of combining actual spatial models in a training VR environment. Nevertheless, the proposed simulator is expected to offer the following advantages:
• Representing scenarios based on the model of actual complex construction sites through which the workers can familiarize themselves with the site.
• Providing scenarios with several pieces of equipment operated by several trainees simultaneously.
• Capturing the realistic behavior of the surrounding equipment, where trainees can be exposed to safety-specific education [7].
There are some limitations in the presented Project:
First, due to the lack of relevant actual site data about equipment movement. Second, the necessary parameters for feedback on safety and productivity performances are not incorporated in the presented prototype.
Finally, the case study was only tested for a single trainee. Based on the above limitations, the future work of this research will pursue the following:
• Collecting data of equipment tracking from actual sites that can be simulated in parallel.
• Improving the prototype to provide necessary feedback on safety and productivity performances.
• Conducting a comprehensive case study with multiple users [9].
REFERENCES
[1] Boyanovsky, H., Imagining the future for hydraulic excavators, SAE OHE 100:Future Look, pp. 77.
[2] Zubko, N., Heavy rotation - a new generation of hydraulic excavators: better, stronger, faster, Utility Contractor, pp. 19–23, 2007.
[3] Roth, M., 2010, Earthmoving's new frontier, Rental Equipment Register, Available:http://rermag.com/trends_analysis/interviews/rer-interviewsearthmoving-equipment-manufacturers-20100401/index2.html,9 February, 2011.
[4] Engel, S., Alda, W., and Krzysztof, B., Real-time computer simulator of hydraulic excavator, Proceedings of the 7th Conference on Computer Methods and Systems, Krakow, Poland, pp. 357-362, 2009.
[5] Fukaya, K., and Umezaki, S., 2002, "Development of Excavator Simulator Using Virtual Reality Technology," Virtual Reality Society of Japan Annual Conference.
[6] Akyeampong J , Udoka S and Park E 2012 A Hydraulic Excavator Augmented Reality Simulator for Operator Training Proceedings of the 2012 International Conference on Industrial Engineering and Operations Management Istanbul, Turkey, July 3 – 6
[7] Haddock K 2007 The Earthmover Encyclopedia Motorbooks-MBI Publishing Company: Minneapolis, Minnesota, USA
[8] Nainggolan F L , Siregar B and Fahmi F 2016 Anatomy learning system on human skeleton using Leap Motion Controller In Computer and Information Sciences
(ICCOINS), 2016 3rd International Conference on (pp 465-470) IEEE
[9] Nowicki M , Pilarczyk O , Wąsikowski J and Zjawin K 2014 Gesture Recognition Library for Leap Motion controller Poznan University of Technology: Poznań
[10] Tao N , DingXuan Z , Yamada H and Shui N 2008 A low-cost solution for excavator simulation with realistic visual effect In Robotics, Automation and Mechatronics, 2008 IEEE Conference on (pp 889-894) IEE
Comments