P-TRAP: a Panicle Trait Phenotyping tool
© AL-Tam et al.; licensee BioMed Central Ltd. 2013
Received: 19 February 2013
Accepted: 24 August 2013
Published: 29 August 2013
Skip to main content
© AL-Tam et al.; licensee BioMed Central Ltd. 2013
Received: 19 February 2013
Accepted: 24 August 2013
Published: 29 August 2013
In crops, inflorescence complexity and the shape and size of the seed are among the most important characters that influence yield. For example, rice panicles vary considerably in the number and order of branches, elongation of the axis, and the shape and size of the seed. Manual low-throughput phenotyping methods are time consuming, and the results are unreliable. However, high-throughput image analysis of the qualitative and quantitative traits of rice panicles is essential for understanding the diversity of the panicle as well as for breeding programs.
This paper presents P-TRAP software (Panicle TRAit Phenotyping), a free open source application for high-throughput measurements of panicle architecture and seed-related traits. The software is written in Java and can be used with different platforms (the user-friendly Graphical User Interface (GUI) uses Netbeans Platform 7.3). The application offers three main tools: a tool for the analysis of panicle structure, a spikelet/grain counting tool, and a tool for the analysis of seed shape. The three tools can be used independently or simultaneously for analysis of the same image. Results are then reported in the Extensible Markup Language (XML) and Comma Separated Values (CSV) file formats. Images of rice panicles were used to evaluate the efficiency and robustness of the software. Compared to data obtained by manual processing, P-TRAP produced reliable results in a much shorter time. In addition, manual processing is not repeatable because dry panicles are vulnerable to damage. The software is very useful, practical and collects much more data than human operators.
P-TRAP is a new open source software that automatically recognizes the structure of a panicle and the seeds on the panicle in numeric images. The software processes and quantifies several traits related to panicle structure, detects and counts the grains, and measures their shape parameters. In short, P-TRAP offers both efficient results and a user-friendly environment for experiments. The experimental results showed very good accuracy compared to field operator, expert verification and well-known academic methods.
There is a wide range of rice panicle architecture among varieties concerning the number and order of branches, and axis elongation. Natural inter-specific and intra-specific variations in morphological traits represent a largely untapped highly valuable resource for genetic improvement by breeding. For efficient selection of beneficial alleles for breeding, natural variation needs to be well characterized at the phenotypic and molecular genetic level. In addition, the study of natural variation is also important to understand the evolution of morphological traits and the molecular genetic mechanisms underlying them. To exploit the diversity of rice panicle resources, panicle morphological traits need to be identified and quantified. Plant phenotyping involves screening large collections of accessions to facilitate the discovery of new interesting traits, and analyzing known phenotypic data to identify the genes involved in their diversity, to be able to use these genes in plant breeding. To collect these data, the usual procedure consists in laborious manual measurements on predefined traits such as panicle length, the number of branches, the order of branches, the number of grains, and grain size. Depending on the degree of complexity of the panicle, manual phenotypic analysis is time consuming and it is impossible to evaluate and quantify all traits (such as branch and spikelet positions in the panicle) to obtain an accurate overview of panicle architecture. Moreover, manual phenotyping is often destructive for the plant making it impossible to use the same panicle to measure other traits. Given the importance of gene discovery and crop improvement, there is thus an urgent need to automate such tedious and time- consuming tasks. The development of an easy high-throughput panicle phenotyping method should aim to standardize the measurement and extraction of panicle traits. In recent years, plant phenotyping research has led to the development of software for plant screening facilities. Recent image processing solutions, such as TraitMill and HTPheno, offer general analysis for the measurement of plant height, volume and colorimetry [5, 6].
Other software provides 2-D image-based semi-automated processing for leaf phenotyping (Phenopsis or LAMINA) and root data monitoring (GROWSCREEN) [7–9]. Specific rice image-based solutions have been developed for phenotyping and involve the measurement of parameters such as grain size (length, width, and thickness), panicle length, and the number of tillers [10, 11]. However, these methods could not be adapted to rice panicle structure phenotyping and require expensive equipment. Ikeda et al developed a software named PASTAR (Panicle Structure Analyzer for Rice) and PASTA Viewer, to automatically extract values for length, number of branches, and number of grains from scanned panicle images. However, this software is under license, thus limiting access by the scientific community. Recently, a program named Smartgrain was developed to quantify seed shape and size. However, this software does not process the grain attached to panicles but only individual grains . In this context, it was important to develop an easy-to-use freely available open-source software based on 2-D image processing for the analysis of rice panicle structure.
Here, we propose a Java-based stand-alone application named P-TRAP (for Panicle TRAit Phenotyping) to easily quantify 2-D panicle traits. The labor-intensive processing is automated but post-processing options allow users to improve the quality of the analysis using their expert knowledge. The proposed pipeline has different tools: a tool to analyze panicle structure, and a spikelet/grain counting and shape analysis tool. The software allows automatic detection of the structure of the panicle from a spread panicle image consisting of different morphological traits that are not easily accessible through manual phenotyping. The spikelet/grain counting option detects the grains on the panicle, counts them and quantifies different shape parameters. The novelty of P-TRAP is the simultaneous analysis of panicle structure and grain counting/shape on the same image. These shape parameters can also be measured from images of spread seeds. The interface allows the two analyses to be performed at the same time (or separately) and extracts the different traits in different output formats (CSV and XML) to facilitate data analysis and access to OpenAlea platform facilities . In this study, we used this program to analyze the panicle structure of various accessions of O. sativa,O. glaberrima and O. barthii and to compare the results with manual measurements to check the robustness of the software.
P-TRAP is written in Java with a user-friendly GUI. The GUI is built on top of the Netbeans Platform (version 7.3), which provides a modular underlay for the system’s architecture. The software provides different features for users to conduct their experiments and edit and collect the final results. It offers an editor for the input image, the panicle structure and the grains. The user interaction is mostly performed by using the mouse or keyboard shortcuts. In addition, developers can easily add new features to the application, as it is very modular.
In P-TRAP, the user first has to create a project. The source images can be then imported for processing using the GUI. The project can contain one or several images. They can be processed individually or as a batch to support different workflow scales. Basic pre-processing steps can be applied on the images. Cropping and scaling processes are available and can be performed interactively using the GUI.
The quantification of the panicle traits is based on the detection of the structure of the panicle followed by a conversion of the skeleton into a mathematical graph (Additional file 1). The pipeline for converting the image of the panicle to a graph can be described as follows: input image I is converted to grayscale and then a Gaussian blur filter with a kernel of size kernelSize is used to smooth the image. The smoothed image is locally thresholded by using the mean-c local thresholding approach , resulting in a binary image. The blurring filter is used to obtain a smooth binary image, and leads to a skeletal image containing fewer undesirable small spikes . Due to variation in the brightness of the image, small holes may remain in the binary image. Unless these holes are filled, corresponding cycles may appear in the skeleton, which may cause several problems during the skeleton analysis task (Additional file 1). To solve this problem, small holes with an area ≤ minParticle are filled in to yield a “solid" binary image, I solid.
The I solid image is skeletonized by the Zhang-Suen’s (ZS) thinning method . A major drawback of this method is that the final skeleton may produce staircases, in which case the Holt’s staircase removal method  is applied. A fast lookup-based implementation of ZS method can be found in . To locate the panicle skeleton in the image, all the components in the skeletal image are searched. The biggest is returned as the panicle skeleton. The skeleton is returned as a list of points (skeletonList) that indicate the positions of the pixels of the skeletons in the image. The xy-origin of the image is at the top left of the image. This list is then converted to a graph G which is then cleaned and refined (G refined). Cleaning is based on removing terminal edges whose length is less than a threshold minSpike (default value = 40 pixels, modifiable by the user). An edge is terminal if one of the vertices it connects has one and only one neighbor.
The calculation of the panicle structure traits is based on the mathematical graph produced from the panicle detection task. Quantification includes two main steps: vertices classification and graph quantification.
In the beginning, the primary vertices (white circles) are identified by decomposing the graph at the start-generating vertex (main root of the graph) into a set of subgraphs. Therefore, each neighbor of the main root is a root of a subgraph. Among the roots of the subgraphs, the one that belongs to the “heaviest” sub-graph is chosen as the“winner” vertex and then classified as primary (Figures 4b, c). The other roots are classified as secondary (i.e. one level lower). Similarly, the heaviest sub-graph is decomposed at its root into sub-graphs, and the new winner is classified as primary, and so on, until the end generating point is reached (Figures 4d-f).
The remaining unclassified vertices are classified in the same way as the primary ones. At each secondary vertex (that has an unclassified neighbor), its parent sub-graph is decomposed and the winner vertex is classified as secondary. The other losing vertices are classified as tertiary and so on (Figure 4g). The classification finishes when all the vertices in the graph are classified (Figure 4h). The graph terminology is defined as follows: Primary Axis (PA) is the main axis of the panicle (i.e. the panicle rachis), Secondary Axis (SA) is a branch attached directly to the PA (i.e. corresponds to a primary branch of the panicle), Tertiary Axis (TA) is an axis attached to a secondary axis (i.e. corresponds to a secondary branch of the panicle); Quaternary Axis (QA) is an axis that is attached to a tertiary axis (i.e. corresponds to a tertiary branch of the panicle).
Structural and grain related traits of a rice panicle
Panicle structure traits
Spikelet (Grain) traits
Primary Axis - length
Spikelets - number
Primary Axis - diameter
Spikelets - length
Secondary Axis - position
Spikelets - width
Secondary Axes - number
Spikelets - area
Secondary Axis - length
Spikelets - perimeter
Number of Nodes
Spikelets - circularity
Secondary Axes Intervals Length
Spikelets - compactness
Tertiary Axes - number
Spikelets - ellipticity
Tertiary Axes - length
Aspect - ratio
Tertiary Axis - position
Tertiary Axes Intervals Length
Quaternary Axes - number
Quaternary Axes - length
Quaternary Axes - position
Quaternary Axes Intervals Length
Finally, the panicle diameter or primary axis diameter (PA_diameter) is found by calculating the Euclidean distance map (EDM) of the I solid binary image using an efficient algorithm described in . In EDM images, each pixel has a value that defines the radius of the maximum ball (the maximum distance from this pixel to the image background). A circle with a small radius centered at the start generating point is defined as a search area. The PA_diameter value is then estimated as twice the square root of the maximum pixel’s value in this predefined search area.
where σ(I binary ∘ d m ) is the standard deviation (STD) of the particle area.
By applying a brute-force algorithm for all disks in the range [d min,d max] with step parameter of 1, the optimal disk is the one with maximum Θ in this range. In (2) if n P < n min, where n min is a small integer, Θ is not considered. Θ is maximized, when n P is big and σ is small, which implies an adequate disk size and consequently an appropriate grain size. Once the adequate disk size is determined, the perfect grain size is just the median of the particles in the binary image opened at this disk size. The median is chosen because it has a good gross-error toleration ratio and 50% breakdown point . At this point, the first mathematical opening level is finished, with the perfect grain size and the optimal disk size identified.
where p i is the particle and is the length of the arc inside p i . In this work, a contour point is termed concave if its concavity ≥ 0.6.
The previously described method of grain detection is designed to detect both spread out and clustered grains. However, spread out grains without the panicle can be detected without all the computation involved in the proposed method. In this context, we used a simpler pipeline (similar to the one used in ) just for the detection of spread out grains. Basically, given a binary image I binary, a mathematical opening with a small-predefined disk kernel can be used to remove the juts from the seeds and to smooth the contour of the grain. The grain traits listed in Table 1 are found as the following:
Where Δ(.,.) is the Euclidean distance metric.
Width: For any two points in the contour, the width is the length of the longest line perpendicular to the length’s line.
where μ .,. is the central moment of the specified order.
System main modules
Core image processing tasks
Basic mathematical tasks required by other modules
Performs the graph tasks
Particles processing and quantification
Manages the rice project folders and files
Manages the application options and the algorithm parameters
Responsible for creating user friendly widgets for elegant user interactions
The main module for connecting the user commands and the core modules
Generates the reports
Manages the file system
The data collected from the processed images are exported in two different formats: XML and CSV. The XML format is used to store the panicle structure and the grains particles, and can be exported for other applications such as OpenAlea after conversion to the MTG (Multiscale Tree Graph) format . Each analysis run produces two files: .ricepr and .ricegr for the structure and the grains, respectively. More information on the structure of the output files is available in Additional file 2. A CSV file is also generated to allow direct visualization of the results and easy transfer to spreadsheet software (e.g. Microsoft Excel). The results of the quantification of the panicle and grains are stored in files with two different levels of details. These CSV files are:
MainTraits.csv: contains the main general data about the panicle.
GrainsTraits.csv: contains the average values of all the data on the grain’s traits.
AllTraits.csv: contains detailed data on the traits of each branch.
In addition to GrainsTraits.csv, each image has a result file that describes each grain trait individually in the Particles folder.
To evaluate the accuracy of P-TRAP, 26 different images of panicles from O. sativa, O. glaberrima and O. barthii were tested in both structure and grain counting tasks. Images were captured using a digital camera (Sony DSC-W55) and saved in the JPEG format (Joint Photographic Experts Group). Image size was 2592 × 1944 with 72 dpi (see Additional file 3 for the 26 images tested in this work).
For grain detection and quantification, either RGB images of spread out panicles or images of spread out seeds without panicles can be used (Figure 2). Specific images of spread ou seeds have been captured using a digital camera (Canon PowerShot G12) with a size of 3648 × 2048 at 180 dpi in the JPEG format (see Additional file 4 for the images that were tested in this work).
The structure finding used by P-TRAP was evaluated and tuned by an expert using the obtained graph, and the options from the GUI, and was compared with results obtained by a field-operator created results (FO). The grain counting method was compared to two academic methods, a Lab Counting (LC) and the FO. The parameter values (in pixels) used for the tests were minSpike = 40, kernelSize = 3 × 3 and minParticle = 1000 for image to graph conversion. For grains counting, d min and d max were set to 5 and 14 pixels, respectively.
Results of panicle structure quantification
P-TRAPv Corrected P-TRAP (%)
Corrected P-TRAP v FO (%)
Considering all the measured traits, the average deviation between the P-TRAP automatic results and the corrected ones after expert evaluation was 2.68%, with deviations ranging from 0.25% to 6.09%(Table 3). Overall, these values indicate that P-TRAP provides robust detection and quantification of panicle structure traits with only a little post-processing required by the user. The average deviation between the corrected P-TRAP results and the FO was 6.27% (deviations ranged from 2.06% to 12.14%). The higher deviations were caused by the nodes and the number of tertiary axes (Nodes nb and TA nb respectively in Table 3). The high deviation observed in the values of TA_nb after expert evaluation compared to FO might be due to the fact that panicles are sometimes not properly spread out and branches overlapped (as illustrated in Additional file 5). This problem can easily be corrected by spreading the panicle out better. Furthermore, the panicle images are fixed to the background by metal pins. In some cases, the pins falsify the elongation of the branches. The difference in the number of nodes observed between corrected P-TRAP values and FO might be related to a difference in the evaluation of this feature between the software and the field operator. In the software, each secondary axis is born by an individual node. However, in some accessions, secondary axes are born by the same node and the field operator considered these as a single node.
Compared to the FO, raw P-TRAP results are more than 90% correct, which is acceptable for this difficult problem. These comparisons indicated that P-TRAP provides reliable quantification of the panicle traits as long as the panicle is properly spread out against the background.
Grain counting results
P-TRAP v FO
P-TRAP v LC
WS v FO
WS v LC
CSS v FO
CSS v LC
An average deviation of 7.44% (deviations ranged from 0.86% to 30.88% depending on the image with a standard deviation of 6.21% and a positive deviation sign) was observed between raw P-TRAP results and FO. The results also had a deviation of 6.84% from LC (deviations ranged from 1.80% to 26.1% with a standard deviation of 4.37% and a positive deviation sign). In contrast, the WS approach had an average deviation of 11.11% (deviations ranged from 1.85% to 39.29% with a standard deviation of 8.87% and a positive deviation). Regarding LC, WS had an average deviation of 10.84% (deviations ranged from 1.53% to 36.84% with a standard deviation of 7.60% and a negative deviation sign). The CSS method had an average deviation of 10.76% (deviations ranged from 0% to 32.35% with a standard deviation of 9.13%, and a negative deviation sign). The comparison of CSS and LC had an average deviation of 10.32% (deviations ranged from 0.48% to 26.09% with a standard deviation of 7.71%, and a positive deviation sign). P-TRAP outweighs all other methods and produces the lowest deviation and standard deviation. Which ensures the stability and accuracy for when tested to different panicles with different type of grains. The WS method is widely known to be efficient in segmenting overlapped circular shapes , but under-segments elliptical shapes when the overlap ratio is high . Furthermore, the watershed over-segmentation problem can be clearly observed when the contour is noisy (Additional file 6). Although the CSS method is slightly better than the WS approach, in this context, it was difficult to set up an overlapping threshold for the grains that copes with the variation in the grains. The parameters used for this method were samplingFactor = 3, saddleHeight = 2, overlappingFactor = 0.7.
In overall, the P-TRAP method gave a good estimation of the number of grains on the images tested. It was efficient in finding the “optimal” disk size for mathematical opening. The difference between P-TRAP and FO may be a consequence of the overlapping of the grains which makes it difficult to estimate the exact number of grains (Additional file 5). Nevertheless, in contrast to other methods or applications, the used method has the advantage of detecting and counting grains directly on the panicle images. In addition, P-TRAP has different options that can be adapted to work with color and grayscale images (Additional file 2).
Finally, grain traits (Table 1) were measured using the same set of images of spread out panicles in addition to 21 images of spread out seeds (see Additional file 4). Additional file 6 presents two different examples of the detection of grain traits performed by P-TRAP from the two types of images. Averaged values of seed traits in output files result from individual seeds (i.e. seed clusters from spread out panicles or spread out seeds were not considered for analysis). The ability of P-TRAP to detect and quantify seeds directly on spread out panicles makes it possible to analyze seed shape traits in relation to their position in the panicle. In this context, and in comparison to the only available closed-source application (Smartgrain), both P-TRAP and Smartgrain have pros and cons. Smartgrain has two methods of segmentation, color and grayscale. The color segmentation method needs the user to define the grain and background colors. The grayscale segmentation method has a problem. It is so sensitive to variations in lightness (Additional file 6). These shortcomings can be obstacles if the source images are grayscale and have a small illumnation variation. In contrast, detection of grain traits is one of the three main tasks P-TRAP offers. Concerning segmentation, P-TRAP uses a local adaptive thresholding method (mean-c) for grayscale images. For colorful images, P-TRAP also provides an option which, like Smartgrain, asks the user to select the grain and background colors. However, as mentioned above, the main advantage of P-TRAP is that it can detect grain traits on the branches while Smartgrain does not have this feature.
Some of the requirements for P-TRAP were user-friendliness, multiple platform support, extensibility, and compatibility with other plant inflorescences with similar structure. P-TRAP also uses some general methods used for image processing. The challenge in detecting structure was to convert the panicle to a graph, and to quantify it. It has been shown that the thinning step is very important in obtaining the structure of the objects in binary images. Many applications depend on the skeletonization process to minimize the amount of data to be processed, e.g. Quench function , to extract accurate features for image matching , to perform image warping , or to analyze plant root structure .
The skeleton was efficient in revealing the structure of the panicle, but not enough to accurately quantify the panicle. Therefore, the panicle skeleton was converted to a mathematical graph, which was more flexible. Graphs and contours are very efficient to deal with skeletons -. In many cases, the skeleton contains small insignificant branches and cleaning has to be applied to clean the skeleton and preserve its structure at the same time. Different spike pruning approaches are available, such as the distance transform , the number and distribution of the maximal disk , branch length  and so on. In this work, the skeleton was initially converted to a graph and the graph was then cleaned by removing all spikes that were shorter than a threshold minSpike (an editable parameter in the P-TRAP options). In addition, the single-grained branches in the panicle were not significant and had to be removed. Skeleton processing, graph processing, and the quantification methods are implemented in independent modules so any improvement or extension to any of these processes can be made very easily. In addition, these modules can be reused in other projects. Concerning the detection and quantification of the grains, the challenge was to directly detect the grains on a panicle with overlapping grains and variations in size. The detection approach, particle analysis, and the central moments are implemented as modularly as possible to allow for future extensions and re-usability.
P-TRAP, a freely available application for processing plant panicles is described here. This tool will be very useful for exploiting the rice diversity resources and for categorizing rice in different groups, based on inflorescence phenotyping. The tool can be used for analysis of architecture (relationship between different morphological traits), for analysis of genetics (both forward and reverse approaches) and for breeding programs. Moreover the ability of P-TRAP to detect and quantify seeds directly on spread out panicles makes it possible to analyze seed shape traits in relation to their position on the panicle (i.e. the apico-basal axis, primary branches vs. other branches).
The rice inflorescence varies widely among accessions and species in terms of branching structure and seed shape. The development of software able to automatically extract quantitative values of panicle structure and seed traits will facilitate the phenotyping of these morphological traits. A complete framework for analyzing rice panicle images is proposed in this paper. The application provides several editors for the input image, the detected structure, and the grains. The structure quantification method was compared to a manually created ground truth and the results showed an accuracy of about 90%. Grain detection and the counting method were compared to two academic methods as well as to ground truth and P-TRAP outperformed the other methods. However, the application, especially the method for detecting the skeleton of the panicle and converting it to a graph has one main shortcoming. It may not correctly detect overlapped branches, and in some cases, this may require some manual post processing to correct the structure. However, this problem can be minimized by carefully spreading out the panicle on the background. On the other hand, P-TRAP can efficiently deal with different rice panicles regardless of their size or complexity. Finally, the P-TRAP processing pipeline is implemented in a highly modular environment and developers can easily improve the application. A further important feature of P-TRAP is that the data are stored in XML files, which can be used in other applications such as OpenAlea, a platform dedicated to plant architecture.
In addition to P-TRAP’s fully featured GUI, some other features are:
Free open source application
Written on top of a well-known modular platform (Netbeans Platform)
Allows the users to save the processed image.
The application comes with different installers that are available at the application’s website. The source code and a sample project can be found in Additional files 7 and 8, respectively. For details of the GUI features, the reader should refer to the user manual in Additional file 2.
Project name: P-TRAP
Project home page: http://bioinfo.mpl.ird.fr/index.php?option=com_content&view=article&id=102&Itemid=2.Several video tutorials can be found at this URL.
Operating system(s): Platform independent
Programming language: Java
Other requirements: JRE ≥1.6 to run the application. To compile the source code, the Netbeans Platform ≥ V.7.3 IDE, Java Matrix Package (JAMA) ≥ V.1.0.2 and Java Advanced Imaging (JAI) ≥ V.1.3 libraries are needed.
License: GPL V3
Any restrictions to use by non-academics: As specified by GPL V3 license.
Comma separated values
EXtensible markup language
Multiscale tree graph
Center supported segmentation
Field operator result
Panicle tRAits phenotyping
Panicle structure analyzer for rice
Java matrix package
Java advanced imaging
HR Shahbazkia would like to thank D Fontenille (head of UMR MIVEGEC, in IRD, Montpellier France) for the reception facilities. The authors would like to thank Joe Tohme (CIAT, Cali, Colombia) for the field phenotyping facilities and for the production of spread out panicle images. Harold Chrestin is thanked for the spread out seed images. This work was supported by IRD institutional funding, GRiSP funding (MENERGEP project), Agropolis fonudation and Cariplo foundation co-funding (FIRST/EVOREPRICE 2011).
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License ( http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.