Introduction

There is a wide spectrum of applications for simulated human motion, from entertainment to clinical studies. In particular, gait analysis, jumping, standing, and other movements have been simulated and analyzed by various groups in physiotherapy, sport and rehabilitation research.^1-3 In total knee replacement surgeries, it has been demonstrated that malalignment causes increased wear of the implant, decreases longevity of the construct, and alters kinematics of the knee.^4-6 Simulations provide us insight into the intricate working of the musculoskeletal system, and augments our knowledge from experimental measurements. Such techniques may prove particularly useful in personalized medicine and surgical planning.

To manipulate or to pose the computational model of a patient from computed tomography (CT) or magnetic resonance imaging (MRI) scans, researchers may employ a physics-based method or a geometry-based technique. In the finite element method (FEM)–a physics-based method, the body is divided into small volumes.⁷ Mechanical properties are explicitly specified for each of them and they are approximated as springs. A model called Adrienne is capable of realistic anatomical motion of muscles, fat, and skin using FEM.8 However, FEM calculations are complex to set up and computationally expensive.

On the other hand, geometry-based methods are relatively easy and computationally efficient. They are used extensively in the animation industry. A geometry-based method is called subsurface deformation (SSD).^11-12 Under the scheme, a character for animation is represented by a surface mesh corresponding to its external appearance. The posing of the character is controlled by an invisible rig or armature underneath. Since a CT or MRI scan can be converted into polygonal meshes to represent the body and tissue surfaces, many groups have successfully created patient-specific models in various postures, for example, the movement of the patella⁸, a full-body model from the Visible Human Project⁹, a full-body model from MRI that can be arbitrarily posed¹⁰, and posing hand models from CT¹¹ and MRI¹².

Our work built on the simplicity of SSD. We aimed at defining and evaluating a method for creating an accurate posable model of the knee joint. Important anatomical and biomechanical features of the knee were incorporated into our model.

Materials and methods

We used a 3D animation software Blender (www. blender.org) to animate the human lower limbs from BodyParts3D library¹³ (http://lifesciencedb. jp/bp3d/). BodyParts3D version 4.3 meshes of the skeletal tissues of the lower limbs in Wavefront object (OBJ) format, one of the many file formats that store 3D object data, were downloaded from the website and imported into Blender. The object’s surface was represented by a polygonal mesh of interconnected vertices which were 3D locations sampled on the object surface. Each polygon defined a small patch of the surface. Thus, the mesh comprised of non-overlapping polygons covering the entire surface.

Modeling the cartilages, the menisci and the muscles

We derived cartilage models from the femoral condyle and the tibial plateau surfaces. The femoral surface was thickened by 2.005 mm for the femoral cartilage,¹⁴ while the tibial one was thickened by 1.6 mm for the tibial cartilage.¹⁵ The result is as rendered in Figure 1. The cartilages and menisci were attached to the rig rigidly so that the femoral cartilage followed exclusively the thigh bone while the tibial cartilage and menisci followed exclusively the lower leg bone.Our work was carried out in several steps. First, modeling the cartilages, the menisci and the muscles was completed. Because these data were not available from BodyParts3D, we designed them inhouse and attached them to the rig. The rotation axis of the knee was identified, followed by designing of the anatomy-driven armature. The lower limbs were then attached to the rig through skinning. Furthermore, we quantified the contraction of the muscles and the overlapping of different tissues during flexion of the leg.

Please refer to pdf document for figure 1.

To study the interaction between muscles and the skeleton, we modeled three muscle groups, including the quadriceps femoris in the anterior compartment of the thigh, the muscles of the posterior compartment of the thigh including biceps femoris, semitendinosus and semimembranosus muscles as well as the gastrocnemius muscle in the posterior compartment of the leg. Muscles and their tendons were modeled as one object.

Identification of the axis of rotation of the knee

To find the rotation axes in the 3D visualization of the data, we first identified five bony landmarks and then the five anatomical axes. The five bony landmarks were the geometric center of the hip joint, the center of the knee joint, the center of the ankle joint, the subchondral line of the tibial plateaus and the line of the surface of the femur condyles. The five anatomical axes were the anatomical axis of the femur, the anatomical axis of the tibia, the mechanical axis of the femur, the mechanical axis of the tibia and the mechanical axis of the lower limb. Their alignments were checked against the literature¹⁶ before rigging.

We used the flexion-extension (FE) axis as the axes of rotation. It was defined through the femoral mechanical axis and the transepicondylar (TE) axis using the “cube” method.¹⁷ Figure 2 shows the cube method and axes as rendered in Blender. The mechanical axis is the line joining the center of the femoral head and the apex of the femoral notch (line A in the figure). The TE axis is the line connecting the widest two points on the condyle – the sulcus on the medial femoral condyle and the eminence on the lateral femoral condyle. Thus, a cube was drawn parallel to the mechanical axis (object E in the figure). It was gradually expanded to cover most of the distal head of femur except the two farthest points (D on both sides in the figure). These two points were then connected to form the TE axis (line B). Based on the data about the angular deviation between FE axis and TE axis18, the FE axis was drawn in our model (line C).

Please refer to pdf document for figure 2.

After the FE axis was defined in the model, an empty object was created such that its local x-axis aligned with the FE axis. This empty object was then available as a new rotation system in subsequent rigging of the armature system (Figure 1).

Designing the anatomy-driven armature

The armature consists of a hierarchical set of interconnected rig bones. Each rig bone is associated with a 3D transformation (position, scale and orientation). The hierarchy stipulates the ways that the 3D transformation from a parent bone upstream are transmitted and combined to that of a child bone downstream. Thus, moving one rig bone can affect the many others, like raising the thigh would move the entire leg. The hierarchy of our armature system (pelvis → thigh → lower leg → foot) is shown in Figure 3a.

Please refer to pdf document for figure 3a.

The posing of the rig bones can be achieved through forward or inverse kinematics. Forward kinematics is the posing of a parent bone, that will also move the child bones together, without affecting their relative orientation to each other. The entire downstream chain in the hierarchy moves as a single rigid object. On the other hand, inverse kinematics (IK) attempts to solve the bone orientations up the hierarchy chain, i.e. when a child bone is posed, for example, dragging the foot bone in IK will flex or extend the entire leg. Since there are infinite solutions for these orientations in 3D, IK rigging requires a pole vector to limit the possible solutions. The pole vector specifies the plane of rotation for the joint.¹⁹

The three joints of the thigh rig bones are located at the center of the hip joint, the empty FE axis defined in the previous section, and at the center of the ankle. We did not model the bending of the feet nor the toes in this work. IK was applied to flex or extend the lower limbs. We designed two control bones to demonstrate the lifting of the leg and squatting. To implement the roll back movement of the knee bending, we chose to use the Blender action constraints, that allows one to control an action using the transformations of another object.

Skinning is the attaching of the surface mesh to the rig. Every vertex in the mesh is associated with every rig bone through a set of values called vertex weights. A vertex weight is a scaling factor between zero and one representing the degree a vertex, a point on the surface, is moved by the movement of a rig bone, wherein a value of zero means that the vertex is not moved by the rig bone and one means rigid following. When a transformation is applied to a rig bone, the surface mesh is also transformed according to the vertex weights. Furthermore, each rig bone affects mostly the local portion of the surface mesh, and its influence decreases with distance.

The surface mesh is also subjected to extra transformation conditions to cause the bulging or inflation of the muscles during flexion-extension. Further details of these conditions can be found in the literature.²⁰

Bone collision

We assessed qualitatively, by visual inspection, and compared our anatomy-driven armature with a conventional one. The inspections included assessing which bones and cartilages were in collision and whether the collision volumes were visually similar on the lateral and medial sides of the joint. Quantitatively, we evaluated the collision of the bones and muscle volume changes during flexion-extension. Collisions were tested using Boolean intersection under the Blender “modifier” which created a “collision” object. The volume of this object was next measured to assess the amount of overlap between the two meshes. The second test was measuring the muscle volume in various poses. All volumes were given by Blender automatically. To the best of our knowledge, we are the first group to carry out these evaluations in an animation.

Results

Movements

Posing of the limbs can be achieved by moving the control bones. Moving control bone, A, would lift of the right leg (Figure 3b), and B, would lift the left (not shown). Control bone C posed the skeleton in a squatting position (Figure 3c). When the muscles were also displayed, we observed that the muscles contract and relax as the skeleton is posed (Figure 4).

Please refer to pdf document for figure 3b, 3c and 4.

Bone collision

Figure 5 shows the severity of the collision between the femur and the tibia during flexion with a conventional armature. All objects in the model, including the bony parts of the femur and that of the tibia, collided with each other. For example, the menisci protruded into the bony part of the femur through the cartilages. On the other hand, our anatomy-driven armature performed better than the conventional method. It did not give rise to collision of the bony parts of the femur and the tibia with other objects. The collision was limited to the cartilages and the menisci. The collision volume with the medial meniscus was estimated to be between 0.02 cm³ and 0.78 cm³ whereas that with the lateral meniscus ranged from below 0.01 cm3 to 0.36 cm³.

Please refer to pdf document for figure 5.

Conservation of muscle volume

The volume of our muscle models in the standing position (180o between the anatomical axes of the femur and the tibia) were 1446 cm³ and 527 cm³ for the quadriceps and the gastrocnemius muscles respectively. Figure 6 shows the percentage volume changes at various states of flexion of the knee joint. The percentage depended on the angle of flexion, and the more the knee is bent, the larger is the muscle volume change. At 70°, the changes were 3.8×103 % for the quadriceps and 1.2×102 % for the gastroceniums muscle. In all cases, the muscle mass was conserved to within 0.01 %.

Please refer to pdf document for figure 6.

Discussion

For modeling concerning the knees, the 3D rotation axis is of paramount importance²¹ and was our main objective in this study. We chose the BodyParts3D data for this purpose. First, the data were derived from the clinical scan of a patient, and was also modified by artists for anatomical displays. It was thus a deliberate effort of the artists to align the coronal plane of the patient to the global xz-plane. This arrangement made the data set ideal for testing our rigging of the lower limbs against a conventional method.

There are many online tutorials on conventional rigging and skinning of animated characters from the Blender website and YouTube. The process we employed was simple and straight forward, similar to the standard ones. It differed from a standard rigging in one aspect, in that our axis of rotation conformed to that of a real skeletal system instead of a global axis. With a standard armature, the pulling or pushing of the foot IK bone will pose the entire leg but it is also necessary for the animator to adjust the pole target bone at the knee sideways, to make the movement more realistic. Our anatomy-driven armature does not require the adjustment of the pole target. Furthermore, the anatomically driven posing was manipulated interactively by users in real time. As the user moves the control bones, the posture changes immediately. This is a huge advantage over FEM calculations.

In this work with Blender, we assumed that the menisci and cartilages were rigid bodies. Thus, it was expected that the movement of the femur would cause collision among objects. However, the anatomy-driven armature performs better than the conventional one visually and quantitatively. Since there was no direct correlation between the collision volume and the flexion angle, we believe that the implementation of the roll back action can be improved. The roll back action is a feature available in Blender to move an object along a predetermined trajectory. It is possible to incorporate this into the movement of the FE axis. It will be part of our future work to study the effect of the roll back action.

SSD also had superior performance when compared to another geometry-based technique called freeform deformation (FFD) in the Korean full body model.¹⁰ Since FFD assumes elastic objects inside the model, the structures inside a posed model are difficult to preserve. The posed model suffered from unrealistic bending of the long bones near the shoulder and knee joints (Figure 9 in reference 10). Furthermore, FFD was not driven by a rig. The control points were manipulated individually, making it inconvenient to pose a human character. With SSD rigging and skinning, the skeleton followed the control armature rigidly. There was no unnatural bending of the bones as a result (Figure 3 and 4).

Please refer to pdf document for figure 9, 3 and 4.

The muscle volume should largely be conserved during contraction to be more physiologically relevant. The measured change is in the order of 106 cm3 per 100 mg of muscle mass in frogs.²² This is between 104 % and 105 % of the muscle volume. We had further demonstrated that muscle volume is conserved to a large degree during flexion and extension, as modeled by Blender. This is an important aspect for some studies that require the tissue volumes. For example, our method can be useful in radiation protection calculations that determine radiation dose to the tissues.²³ Further work is needed to incorporate more soft tissues and other joints into the model.

Conclusion

We successfully exploited the simplicity of Blender and implemented a method to model the articulation of the human knee joint. This pilot study highlighted the ease of application and quantified its errors. Our anatomy-driven rig minimized the collision of skeletal bones during posing and the muscle volume was conserved to within 0.01% of its original value. Thus, our technique is more anatomically and biomechanically accurate than conventional animation modeling. It has great potential in anatomy-driven deformation of clinically derived data from MRI or CT. There are possible applications in patient-specific studies, such as pre-surgical planning and practice, to increase the confidence in surgeons, and in gait analysis in physiotherapy and rehabilitation.