Scientific Challenge


    ICBHI 2019

    Taipei, Taiwan, 17-20 April 2019

  • Important Dates

    Dataset Open Test (Phase I) : September 1 to December 15, 2018

    Abstract submission for SC: Extension to December 20, 2018

    Dataset Open Test (Phase II): January 2 to January 31, 2019

    Notification of acceptance: January 18, 2019

    Paper Final submission: February 20, 2019
    Early registration: until January 25, 2019

    Conference Registration: Before March 31, 2019

  • Scientific Challenge

    Welcome to the second edition of the Scientific Challenge Competition of the Int. Conf. on Biomedical and Health Informatics (ICBHI).


    In this edition of the ICBHI Scientific Challenge, participants are asked to identify the main cardiac systolic time intervals, i.e., the pre-ejection period (PEP) and the left ventricle ejection time (LVET) using ECG and phonocardiogram (PCG) collected in a controlled clinical setting. The pre-ejection period is defined as the time interval between the ECG R-peak and the onset of the opening of the aortic valve (part of the S1 heart sound). The LVET is defined by the time span between the aortic valve opening and closing events. The latter usually corresponds to the onset of the S2 heart sound (see figure 1).

    Figure 1

    Figure 1. Cardiac time intervals in (left) normal heart and (right) heart with systolic heart failure. Legend: Ao – Aorta; LA – Left Atrium; LV – Left Ventricle; PEP – Pre-ejection period; LVET – Left ventricle ejection time (adapted from (Ross et al., 2006))

    The relevance of the main systolic time intervals is related to the fact that myocardial relaxation and contraction are governed by intracellular recycling of calcium ions. The timings of these basic cardiac events are directly related to the health of the cardiac cells and determine the ability of the myocardium to achieve blood delivery according to the metabolic requirements of the organs. Of major importance are the systolic and diastolic timings of the left ventricle, since it is this ventricle’s function to insure the blood flow in the systemic circulation. In the mid 20th century PEP was studied extensively as a measure of cardiac systolic function, whereas LVET was applied as a surrogate of left ventricle stroke volume. A healthy heart exhibits a short PEP and a long ejection time, while myocardial dysfunction prolongs PEP and shortens LVET. This has led to the introduction of the PEP/LVET index to assess the systolic function of the heart. Indeed, a good and significant correlation between PEP/LVET and the left ventricle ejection fraction determined using angiography has been observed among patients with a wide variety of cardiovascular diseases. Furthermore, it was found that in patients with coronary artery disease, the PEP/LVET ratio is a highly reproducible measure that is equivalent in accuracy to the determination of the left ventricle ejection fraction. Despite the introduction of powerful and complete assessment tools of cardiac function such as the echocardiography, a significant interest in STI as a non-invasive tool for left ventricle function assessment persists in several contexts where portable, low intrusive and cheap solutions are imperative, such as in pHealth.


    In order to develop solutions for the challenge, participants will have access to an annotated heart sound database containing snippets of about 30-120s of synchronously acquired heart sounds and ECG in clinical settings collected from healthy and patients with different cardiovascular disorders. Each test data set entry has information regarding beat-to-beat aortic valve events. The goal of this edition’s challenge is to develop a resilient and accurate solution to measure beat-to-beat PEP and LVET timings from heart sounds and ECG (see figure below).

    Figure 2

    Figure 2: Example of synchronously acquired ECG, PCG and echocardiography.

    In order to participate as an individual or a collective team you should first register in order to be able to access the public annotated database of heart sounds and ECG made available for the challenge and to submit your entries to the challenge.

  • Quick Start

    (1) Register using the website of the challenge, you will get an e-mail with training data set link from Scientific Challenge Committee .

    (2) Download the data set that you can use to develop your solution.

    (3) Develop your entry by editing the existing files (Sample entry in MATLAB):

    • Modify the sample entry source code file challenge.m with your changes and improvements.
    • Run your modified source code file on all the records in the training dataset.

    (4) Submit your modified compressed entry file to icbhi2019.sc@gmail.com for scoring. Improperly-formatted entries will not be accepted.

    (5) Submit an overview paper with the best solutions to https://easychair.org/conferences/?conf=icbhi2019 (Please select "scientific challenge" track after log-in)

    (6) Attend ICBHI'19 (April 17-20, 2019) and present your work there (at least one team member must be registered and attend ICBHI).


    For those wishing to compete officially, please follow the additional six steps described in the Rules and Deadlines.

  • Rules and Deadlines

    In this challenge, we adopt the general rules of the Physionet Competition. The challenge is structured in two phases: the unofficial (Phase I) and the official (Phase II) phases. Participants may submit up to 15 entries over both the unofficial and official phases of the competition (see Table 1). Each participant may receive scores for up to five entries submitted during the unofficial phase and ten entries during the official phase. Unused entries may not be carried over to later phases. Entries that cannot be scored (because of missing components, improper formatting, or excessive run time) are not counted against the entry limits.


    All deadlines occur at noon GMT (UTC) on the dates mentioned below.

    All official entries must be received no later than 12:00 GMT on the 20th of February 2019. In the interest of fairness to all participants, late entries will not be accepted or scored.


    To be eligible you must do all of the following:

    1. Register: you should include the full name, email, and affiliation.
    2. Submit at least one entry that can be scored during Phase I until the defined deadline (12:00 GMT, December 15, 2018). All submissions must be in Matlab and be self-contained (e.g. if some special open-source and non-standard toolbox is needed, then it has to be provided; all commercially available toolboxes from Mathworks for MATLAB R2015a are included). Please follow the detailed information on data formats, input and output formats provided in the sample files.
    3. Submit an abstract (up to 1 page) describing your approach on the Scientific Challenge to ICBHI’ 19 no later than December 15, 2018.      


    4. Submit a full paper describing your work for the Scientific Challenge following the ICBHI' 19 rules no later than February 1, 2019, using the regular submission system. Include the overall score for at least one Phase I or Phase II entry in your paper. You will be notified if your paper has been accepted by email from ICBHI’19 until February 10, 2019.
    5. Paper final submission (4 pages) must be received no later than February 20, 2019.
    6. Attend ICBHI'19 (April 17-20, 2019) and present your work there (at least one team member must be registered and attend ICBHI).
    Please do not submit an analysis of this year's Scientific Challenge data to other conferences or journals until after ICBHI’19 has taken place, so the competitors are able to discuss the results in a single forum. We expect the submission of an overview paper with the best solutions to a prestigious Scientific Journal, co-authored by participants of the challenge and taking into account the publications and discussions at ICBHI’19.
    The paper submission page is: https://easychair.org/conferences/?conf=icbhi2019 (Please select "scientific challenge" track after log-in)
  • Scoring

    In case all the necessary files are included in your entry, and they run as expected, your entry is evaluated and you will receive your provisional scores when the test is complete. An automatic email will be delivered informing you of the results. The evaluation will be made on a weekly basis. If you receive an error message instead, read it carefully and correct the problem(s) before resubmitting.

    The overall score for your entry is computed based on the number of cardiac cycles. Let N be the number of cardiac cycles in the test data set. Let PEPi and LVETi be, respectively, the pre-ejection period and the left ventricle ejection time measured by your algorithm for cardiac cycle number i. Furthermore, let PEPiR and LEVTiR be, respectively, the pre-ejection period and the left ventricle ejection time measured using echocardiography (gold standard). The overall score is defined as follows:

    Lower score is better.


    Description of the Database and expected inputs/outputs

    a) Challenge Data

    The database was collected using synchronized phonocardiogram (PCG), photoplethysmogram (PPG) and electrocardiogram (ECG) as well as echocardiography (Doppler mode). The study was carried out with 68 volunteers in the “Centro Hospitalar da Universidade de Coimbra”, Coimbra, Portugal, which are divided into two distinct groups: one containing 33 healthy subjects and another with 35 subjects suffering from various cardiovascular diseases (CVDs), such as hypertension, arrhythmia, acute infarction, AV blocks (pacemaker), angina pectoris, heart failure, ischemia, aortic insufficiency, coronary artery disease and aortic stenosis.


    During this study, the volunteers were requested to stay at the supine position, at rest and after exercise (the healthy subjects), while the ECG (MLII lead configuration), PPG (right hand index finger), PCG (LSB – left sternum border, and apex) and echocardiography (Doppler mode) were collected by an authorized medical specialist. The measurement protocol consisted on the acquisition of echocardiographic data during several runs (at least 4 consecutive runs of 8 seconds), during which the remaining bio-signals were synchronously recorded.


    Using the echocardiographic measurements, a clinical technician manually annotated several cardiac parameters, such as the pre-ejection period and the left ventricular ejection time. The population was not balanced for gender (51 male and 17 female). The biometric characteristics of the population are (mean ± std):


    Age: 29,72 ± 8,54 (Healthy subjects) and 58,97 ± 17,22 (CVD subjects) years

    BMI: 24,48 ± 2,41 (Healthy subjects) and 25,38 ± 3,10 (CVD subjects) Kg/m2



    The parameters available in the data set, collected in a daily basis, are:

    Time series:

    • ECG – runs between ap. 8s (from ECHO) and 400s (from HP monitor)
    • PPG – runs of ap. 400s
    • PCG – runs of ap. 30s-120s
    • Echocardiography – ap. 4-8 runs of 8 secs

    Meta data:

    • Weight
    • Height
    • Body mass index
    • Age
    • Gender
    Cardiovascular disease/condition


    Based on the echocardiographic images a clinical technician annotated several hemodynamic measures, which are listed below:

    • ECG R-peaks – Beat-to-beat
    • Aortic valve opening (AVO) – Beat-to-beat
    • Aortic valve closure (AVC) – Beat-to-beat
    • Mitral valve opening (MVO) – Beat-to-beat
    • Heart Rate (HR) – Beat-to-beat
    • Pre-ejection Period (PEP) – Beat-to-beat
    • Left ventricular ejection time (LVET) – Beat-to-beat
    • Isovolumetric contraction time (ICT) – Beat-to-beat
    • Isovolumetric relaxation time (IRT) – Beat-to-beat
    • Left ventricular outflow tract (LVOT) diameter – Average (onset of the protocol)
    • Left ventricular outflow tract (LVOT) trace – Curve/beat-to-beat
    • Left ventricular outflow tract (LVOT) velocity time integral (VTI) – Beat-to-beat
    • Stroke volume (SV) – Beat-to-beat estimation from LVOT diameter and VTI beat-to-beat
    • Cardiac output (CO) – Calculated beat-to-beat
    • Ejection Fraction – Average (onset of the protocol)

    The dataset is divided in two parts: a public/training set and a private/testing set. The public partition of the database contains 23 sound recordings. The test set is unavailable to the public and will remain private for the purpose of scoring. The sounds were collected from both normal subjects and pathological patients. Both normal subjects and pathological patients include both female and male subjects. The training and test sets have each been divided so that they are two sets of mutually exclusive populations (i.e., no recordings from the same subject/patient are in both training and test sets).

    Please note that many recordings are corrupted by various noise sources, such as talking, coughing, stethoscope motion, respiratory sounds and intestinal sounds.


    b) Input/ouput formats


    To enter the ICBHI challenge, the participants have to develop an algorithm that is able to read both the biosignals signals and the annotation files, providing as output the PEP and LVET measurements for each cardiac cycle, without any user interaction. The algorithm must be implemented in MATLAB. Each team must send the whole files in a compressed format (*.rar, *.zip, *.7z, or *.tar.gz) from your registered email account to icbhi2019.sc@gmail.com. The archive should contain the following files::

    1. AUTHORS.txt, a plain text file listing the members of your team who contributed to your code, and their affiliations.
    2. All the required code that is used for the analysis. This set of files must include a file with the name challenge.m, which is the entry file of the code and has one input: the name of the record.
    3. Your code should output the file results_RecordName.txt, where RecordName is the name of the recording from training data set that was used for the analysis. Each text file should contain a N*2 vector, where N is the number of cardiac cycles in the recording. Row number i should correspond to the pair PEPi and LVETi measured in ms (mili-seconds).

    We will run your code on the testing set and compare the contents of the results_RecordName.txt files that are generated with the ground-truth annotations of the health professionals. If everything runs as expected, you will get an email with the score of your submission.

    In Phase I (Unofficial) your submissions will be tested on a subset of the testing set, while in Phase II (Official) they will be tested on the whole testing set. Thus, the grade for a specific algorithm may be different in each phase.

    You can download the public set and the expected submission files as zip files.

    As organizers of the ICBHI 2019 Conference, in order to comply to the new General Data Protection Regulation (GDPR) we are asking your permission to use your email to send you information, such as call for papers, call for participation, information on Journal Special Issues, etc. related to the conference.
    If you wish to be excluded from our mailing list, please using the following email address: ttmd@cycu.edu.tw to unsubscribe and your email will be immediately removed.
    If you do not reply to this e-mail, you authorize the organizers of  ICBHI 2019  to use your email for conference related communications.