|Home | About | Journals | Submit | Contact Us | Français|
The article presents a selected area of research and development concerning the methods of material analysis based on the automatic image recognition of the investigated metallographic sections. The objectives of the analyses of the materials for gas nitriding technology are described. The methods of the preparation of nitrided layers, the steps of the process and the construction and operation of devices for gas nitriding are given. We discuss the possibility of using the methods of digital images processing in the analysis of the materials, as well as their essential task groups: improving the quality of the images, segmentation, morphological transformations and image recognition. The developed analysis model of the nitrided layers formation, covering image processing and analysis techniques, as well as selected methods of artificial intelligence are presented. The model is divided into stages, which are formalized in order to better reproduce their actions. The validation of the presented method is performed. The advantages and limitations of the developed solution, as well as the possibilities of its practical use, are listed.
Gas nitriding is a thermo-chemical treatment of iron alloys involving the diffusion saturation of the metal surface with nitrogen. The aim of the thermo-chemical treatment is to produce structural changes in the processed material and, thus, to change its functional characteristics and mechanical or physico-chemical properties (i.e., abrasion resistance, hardness, fatigue resistance, corrosion resistance). The process includes several steps: the production of the active saturating elements’ atoms, forming a nitriding atmosphere; the adsorption of free atoms on the surface of the workpiece, including physical adsorption associated with the attraction of atoms and chemical adsorption associated with the formation of the intermetallic layer; thermally-activated diffusion of atoms in the surface layer involving the spread of the molecules of the diffusing substances in the surrounding medium. During the nitriding process, the nitrogen concentration in the surface layer is changed. Diffusing nitrogen creates a zone of chemical compounds composed of pure nitrides and a diffusion zone (internal nitriding zone), wherein the content of nitrides is reduced to the negligible value in the core material. The whole process can be controlled, among other things, by creating the nitriding atmosphere with a certain potential, produced by different techniques and providing it with a specified flow through the retort. In the case of the bicomponent atmosphere, the dilution of ammonia (NH3) with nitrogen (N2) or dissociated ammonia (NH3dis) results in the reduction of the supply of atomic nitrogen on the steel surface, which leads to a reduction of the surface concentration of nitrogen and, consequently, to limiting the growth of undesirable, porous and brittle ε, ε + γ' zones. The effect of the nitriding process is the outer layer of the structure and the phase composition dependent on parameters, such as the temperature, the duration of the process, the type of material constituting the object subjected to nitriding and the nitriding atmosphere composition.
Nitriding is done on materials comprising Cr, V or Mo nitride-creating elements. During the nitriding process, such phases are formed as nitric ferrite, nitric austenite, γ' nitrides (Fe4N) and ε nitrides (Fe2N). The gas nitriding process is carried out in specially-designed machines, which, depending on the type of gas nitriding, may vary the equipment. The typical facility to carry out gas nitriding (Figure 1) consists of: retort with a system of heaters, a dissociator, inlet and outlet gas systems, the fan circulating the atmosphere and the control system. In addition, the machine can be equipped with sensors to allow the control of the processes parameters, including determining the degree of ammonia dissociation and nitrogen potential, and the sensors allowing for the investigation of the process results (e.g., magnetic field sensor ).
Thermo-chemical treatment in the form of gas nitriding is utilized in a number of engine and pump components’ production. The processed parts are, e.g., connecting rods, cylinder liners, pistons, pins and piston rings, gears and shafts . Nitriding is also widely used to improve the tools’ durability, such as tools for plastic forming, forging dies, molds, drawing dies, cutting tools made of high speed steel, drills, taps, milling cutters, etc.
The processing and analysis of digital images is the area of problems related to the processing and interpretation of video signals using information technology. Automatic processing and analysis of digital images is widely used in cognitive  and practical applications and is utilized in many industry sectors . There are many types of digital image processing and analyzing methods, and their essential task groups are: improving the quality of the images, segmentation, morphological transformations and image recognition. Methods for improving image quality  are used to reduce interference associated with non-linear characteristics of optoelectronic transmitters or aberration of lens systems used in vision systems.
The methods of improving image quality include point (anamorphic) and spatial operations. Point operations are characteristic in that the result of the conversion of pixels depends only on the values of the analyzed pixel, without taking into account the relationship with other pixels located in their vicinity. In the case of spatial operations, analyzed pixel values depend on the values of other pixels from a specific neighborhood or environment dependent on the diameter of the so-called local operator. Another class of methods for processing and analyzing digital images is the segmentation process, also known as labelling . The segmentation process operations involve the division of data forming the image in order to increase the efficiency of recognition and the interpretation of the objects therein. Segmentation methods include two basic groups: area and contour. Area segmentation is a division of the image into homogeneous zones, meeting the established criteria of homogeneity (e.g., intensity, color, texture), which typically represent specific fragments of reality mapped to the picture. Contour segmentation is used to determine the boundaries between objects mapped in the analyzed image. Examples of methods of segmentation are: thresholding , region-growing  and watershed transformation . Another group of methods for image processing and analysis are morphological transformations, by means of which the processing of objects’ shapes, which are mapped in the analyzed image, is carried out. This is done through the use of so-called structurizing elements , which are patterns of specific shapes with highlighted center points. Because of the complexity of morphological transformation algorithms, they are divided into: simple (primary) transformations and complex transformations. Simple transformations are characterized by the fact that they cannot be replaced with other morphological transformations. Complex transformations are divided into the first order and the transformations of the second or higher order. The first order complex transformations are characterized by the fact that their action is based on a few basic operations, and the number of occurrences of different types of basic operations is limited to one. Complex transformations of the second and higher order are transformations whose operation is based on a combination of basic operations; the number of occurrences of each basic operations for the second order is two and for higher order, proportionately more. An important set of methods for image processing and analysis are pattern recognition algorithms. There are many definitions of image recognition process. Image recognition by Duda and Hart is a machine identification of significant regularities in a complex and noisy environment . Bezdek defines pattern recognition as a search for graphical data structures , and Tadeusiewicz defines the image recognition as automatic identification of classes, namely recognition of objects and phenomena belonging to the classes . The image recognition process usually consists of two stages. The first stage is a measurement of the value of the characteristic features of the objects recognized in the image.
The second step provides the analysis of the measured feature values to determine the membership of objects characterized by the features to defined classes. Features, by which objects are characterized, may be different for different types of objects; thus, it is not possible to unify them. Pattern recognition methods, divided due to the implementation of the decision making, are shown in Figure 2.
The choice of methods for image processing and analysis and their use depends on many factors, among which a significant role is played by the type of objects represented in the analyzed images and the assumed limitations of processing time. Different kinds of methods are characterized by specific sets of advantages and limitations, determining their practical application. The effective selection of methods to solve the given problem determines to a large extent the achieved results.
The developed solution makes it possible to identify the conditions under which the chosen coatings were manufactured, based on a limited range of input data in the form of digital image. The tasks of this type fall within the area referred to as reverse engineering and are used inter alia for the purpose of discovering the method used to develop certain products. As for the practical aspects, it can be applied in the analysis of the causes of damage to machines and equipment or the investigation of the technology used by competitors. In the developed model, it was assumed that the automatic inference concerning the parameters characterizing the gas nitriding processes and the obtained layers’ properties will be based on metallographic section photos. The structural model of the method for studying the mechanisms of preparing nitrided layers based on automated analysis of digital images is shown in Figure 3.
The general model of formal analysis is as follows:
The input values space XM is composed of: metallographic sections’ digital images, information on the actual size of the mapped structures, information on the method of metallographic sections’ preparation and information about the type of material constituting the core of the analyzed sample. The output values space YM is the layers’ parameters, information on the possible technologies used in gas nitriding and the process characteristics with which it is possible to produce surface layers similar to those that have been analyzed.
Before examining the images and if multi-stage inference is possible, it is necessary to properly prepare input data in the form of digital images of material samples. Photos should be so arranged that they exhibit the characteristics of the analyzed material structures in the best possible way. The metallographic sections should be prepared using techniques that allow visualization of their characteristics, such as etching, evaporation or coloring . Reagents for etching effect primarily the grain boundaries, and grains of different composition are more or less etched. When the etched sample consists of several phases, then they are shown in different degrees, which allows carrying out the automatic identification based on the photos. It is important that the same analyzed structures are prepared by identical methods. Taking pictures of the material samples is possible with the use of metallographic microscopes currently available on the market, with the function of recording images in digital form, which ensures the repeatability of the obtained images. It is also important that the photos are made for the same sample orientation (e.g., the material surface cross-section is on the left side and the core on the right side of the photo). It is essential for further analysis of images, as analytical algorithms require that one specify the direction of analysis (the order of nitriding zones’ occurrence on the pictures).
The main analysis begins with the inference concerning the choice of procedures covering the methods of processing and analyzing digital images to identify nitriding areas and their characteristics. It is necessary in order to obtain the characteristics of the distribution of the brightness of pixels in the images representing metallographic cross-sections. A method of forming these characteristics shows, inter alia, Equation (3). This step is necessary because of differences related to the representation of the nitriding areas, resulting, inter alia, from the methods of sample preparation and the core material, which could affect the obtained picture.
Formally, this stage is as follows:
The records are presented in the generalized (model) form. In individual instances, the parameters in different sets can have different values, e.g.:
Metallographic section preparation parameters:
Core material parameters:
Digital photo’s parameters:
Inference is carried out using the deductive model  based on the bivalent logic. In this solution, the knowledge of the known cases is represented in the form of complex logical rules of evidence and conclusions. The advantage of this is that, assuming that only certain rules will be used (which are reflected in reality), the result will also be devoid of uncertainty. To write the logic rules, Horn clause format was used :
The next step in the analysis is to identify nitriding areas using the procedures chosen at the application stage.
Formally, this stage is as follows:
The analysis is preceded by identifying the ROI (region of interest) , which is a part of the image subject to further processing (Figure 4) and contains important information from the point of view of the task.
After determining the ROI, the procedures related to the process of gas nitriding zones' identification, processes associated with improving the image quality are performed, which include, depending on the method of sample preparation and core material parameters:
Later in the process, the identification of nitriding areas is based on an analysis of the brightness distribution of pixels. In this process, the projection of the intensities of pixels on the axis mapping the distance from the surface to the core of the material is carried out by evaluation of the average levels for each row in the ROI starting from the surface of the material according to the equation:
An example of the pixels’ brightness distribution in an image representing the ROI on the metallographic section of the nitrided sample material is shown in Figure 5.
For the ROI shown in Figure 4, the nitride zone located on the surface of the sample has a higher brightness of pixels with respect to surrounding areas (this zone in many publications is also called a white zone ). For this reason, it is possible to detect this in a figure showing the distribution of the brightness of pixels as a peak, whose width also determines the thickness of this zone.
After identifying the location and the thickness of each zone in the sample material pictures, the forecasting process of the hardness distribution in the zone of internal nitriding is being implemented (if this type of zone has been identified).
Formally, this stage is as follows:
Forecasting the hardness distribution in the zone of the internal nitriding using visual analysis is based on a comparison of the brightness distribution of pixels of the analyzed sample to the pixel brightness distributions of samples made of the same material, for which the characteristics of the hardness distribution are known. Knowledge of these characteristics, i.e., levels of hardness assigned to specific pixel brightness, comes from the hardness tests carried out using the conventional destructive methods carried out on previously-photographed metallographic sections. Hardness is measured for sample points with varying brightness mapped in images. For certain brightness levels multiple measurements of hardness should be made (the average value of hardness is the end result). Research of this type should be done before applying the method developed in the article and should include similar materials as those that are analyzed using the developed method. Metallographic sections of these materials should also be prepared in an analogous manner to the analyzed ones. The larger the collection of empirical data, the higher the chance of finding in the database material similar to that analyzed and subsequently to identify the hardness based on the brightness of pixels. If there is no data available about similar material with a metallographic section prepared in an analogous manner, the hardness estimation is not possible. Hence, the method is heuristic. It was assumed that the average brightness of the pixel intensity values (Figure 6) corresponds to known levels of hardness of the internal nitriding zone.
After determining the distribution of hardness in the internal nitriding zone, another element in the analysis process is carried out: identification of nitriding technology.
Formally, it is as follows:
At this stage, the process is carried out using a similar mechanism as at the stage of the methods of image processing and analysis selection, i.e., using a deductive model and a bivalent logic. The result of inference is a set of potential gas nitriding technologies, using which, it is possible to obtain the outer layers characterized by similar properties to the analyzed sample. The premises and conclusions of rules used at this stage of the analysis include facts describing: nitriding zones (S), phases (F) in particular zones and gas nitriding technologies (G).
Below is an example of a rule for the deductive process of nitriding technology identification:
The knowledge base may include rules describing any gas nitriding technologies, including unregulated or regulated nitriding, with single or multi-component atmospheres, e.g.: NH3, NH3 + N2, NH3 + NH3dys, NH3 + H2.
The last step in the process of analysis is to identify the characteristics of the gas nitriding process.
Formally, this stage is as follows:
In the inference process at this stage, the Mamdani model  and fuzzy logic  are used. The premises and conclusions of fuzzy rules are constructed with the fuzzy sets mapping the nitriding process parameters. A set of rules to analyze from the set of accumulated rules’ knowledge base is selected based on their conformity with the type of sample core material and the type of technology selected in the previous step. The structure of rules for inference concerning the controlled nitriding technology with a NH3 single component atmosphere is as follows:
The inference at this stage is a complex process with fuzzification (the transformation of sharp signals into fuzzy sets using specific functions that were assumed that will form isosceles triangles with vertices corresponding to the sharp values and the sides stretched throughout the range of the corresponding field), inference (determination of resulting complex fuzzy sets in the output values space for nitrogen potential, temperature and time of the nitriding process) and defuzzification (determination of the sharp values from the complex fuzzy sets using the method of the center of gravity ). If the identified nitriding technology includes a multicomponent atmosphere, then the rules are much more complex, because they describe the proportions between the components of the atmosphere, which may change during the nitriding process.
In order to confirm the usefulness of the developed model, verification for the chosen input dataset was performed. These data were images of material samples obtained from gas nitriding processes with varying process parameters. Samples of material before taking pictures were prepared according to established methods (grinding, Nital etching), so that they can provide reference material for the data stored in the system database. There were also collected data on the actual size of the structures represented on digital images of material samples, in order to properly scale them in the image processing and analysis. The database coupled with a knowledge base used by the system contained imaging data, including 22 specimen photos representing fragments of material elements subjected to the gas nitriding processes. Three material samples were chosen for the analysis: W300, W302 and W320 (Figure 7). These steels were chosen due to their widespread use in the machine industry and, thus, demonstrate the usability of the developed methodology.
The verification process takes several steps. The first verification carried out was the process of inference based on the deductive model, concerning automatic selection of analytical procedures consisting of the methods of image processing and analysis.
Of the four developed procedures characterized by certain attributes (the type of core material represented in the pictures, the metallographic section preparation method, the range of pixel brightness in pictures), a procedure was identified, which by the rule knowledge used in the process of inference enables the most effective vision analysis for these cases. This procedure was chosen by means of automatic inference because of the describing rule, joining the sample etching method used and the resulting ranges of variation of pixel brightness.
In the next step of verification, the internal nitriding zone and nitrides zones were identified on pictures of material specimens. This was most evident for the W320 steel metallographic section (Figure 8).
There were thickness of each zone specified, as well as the hardness in the internal nitriding zone. Hardness estimated using the developed method and Vickers hardness determined for each sample are shown in Figure 9.
Deductive inference-based identification of gas nitriding technologies allow one to obtain material modification corresponding to the analyzed samples, using the indicated potential technologies, including the monocomponent NH3 atmosphere, as well as multicomponent NH3 + NH3zdys, NH3 + N2. Knowing the sample preparing technology (monocomponent NH3 atmosphere-ZeroFlow), the theoretical calculations with the actual parameter values were compared. The values obtained theoretically and the actual parameters used to generate the analyzed material modification are shown in Table 1.
The developed methodology to identify parameter values determining the nitrided layers’ manufacturing process, as well as material properties, based solely on automated analysis of images of metallographic sections and selected methods of artificial intelligence, provides a solution to the problem of limited access to expensive research equipment. The presented methodology helps also to reduce the human factor in research by automating the process. The methodology should be treated as a heuristic solution due to the need to provide a priori knowledge used in the process of automatic inference, which may not cover the required subject range in each analyzed case. In companies of a commercial nature, the developed methodology can be used to discover the methods used to manufacture certain products by the competition. Another area of application can be aiding the analysis of the causes of damage to machines and equipment under field conditions without the need for expensive laboratory equipment and long-term studies. Using only the material specimens’ pictures in analysis is the main limitation of the method, which may in the case of a narrow knowledge base lead to failure to obtain a satisfactory result. One possibility to increase the efficiency of the developed solution is to build an extensive knowledge base to use in the process of inference. Another is to consider the possibility of using other methods, e.g., photoelectron spectroscopy , etc. The developed model methodology is open, which means that it can be subjected to further modifications, which aim to increase its effectiveness.
The authors wish to thank the Industrial Research Institute for Automation and Measurements PIAP for covering the costs of open-access publishing of the paper.
The analysis model and experimental design was planned by Tomasz Wójcicki. Tomasz Wójcicki and Michał Nowicki did the experimental work and analysis of the results. All of the authors participated in the writing of the manuscript.
The authors declare no conflict of interest.