PMC:7796058 / 52791-56255 JSONTXT

{"target":"https://pubannotation.org/docs/sourcedb/PMC/sourceid/7796058","sourcedb":"PMC","sourceid":"7796058","source_url":"https://www.ncbi.nlm.nih.gov/pmc/7796058","text":"5.5. Video-Based Risky Behavior Detection\nCamera stream processing is a popular and quick way to detect objects. Human behaviors and actions can be detected as objects from the video frames using a trained deep learning model. For the detection of risky behaviors such as coughing, hugging, handshaking, and doorknob touching, the You Only Look Once version3 (YOLOv3) which is suitable for real-time behavior detection for online video streams, was trained and applied [63,64]. This library classifies and localizes detected objects in one step with a speed of faster than 40 frames per second (FPS). We considered two main types of risky behaviors for COVID-19 indoor transmission: Group risky behaviors (e.g., hugging) and individual risky behaviors (e.g., coughing). Figure 12 illustrates how to train a model for COVID-19 transmission risky behavior detection using YOLOv3.\nIn total, 603 images for coughing, 634 images for hugging, 608 images for handshaking, and 623 images for door touching were used from COCO dataset [62] for transfer learning for the pre-trained model (YOLOv3). These images were taken from free sources found through Google image searches. For labelling objects, a semi-automatic method was applied. Darknet library was also used for training. For individual behaviors, all of the people in images were detected and labelled in a text file whilst the algorithm aggregated intersected bounding boxes of people into a single bounding box. As wrong labels might be generated, the images should be manually checked to correct misclassified objects. For this step 80 percent of the images were selected for training and 20 percent for testing. To increase the accuracy of this model, the configuration in Table 3 was used.\nTo increase training accuracy and speed, a transfer learning process was applied. The base layer is a pre-trained YOLOv3 that uses the COCO dataset for all of the layers of our model except the last. Transfer learning helps with training by exploiting the knowledge of a pre-trained supervised model to address the problems of small training datasets for COVID-19 risky behaviors [65]. To evaluate the accuracy of the model, we tried to check the results for different video datasets by exporting all of the frames for detection under various circumstances for the metrics listed in Table 4.\nAfter studying the outcomes, we found that the “hugging” and “handshaking” classes experienced the highest false negative results compared to coughing as the larger dataset was being prepared for training. It appeared that hugging and handshaking (grouping actions) were more varied in terms of the types of handshaking and hugging. Therefore, training precision could be improved with the preparation of more varied data. Moreover, some of the false positive results for coughing showed that in most cases, moving a hand near the face was detected as coughing, regardless whether it had actually taken place. Furthermore, the number of false negatives increased in a more populated area. Detected touching behavior results demonstrated high numbers of false negative cases. About 75 percent of false negative cases occurred when the predictor incorrectly detected small objects. Therefore, specifying limitations for box sizes and level of confidence for the predictor can reduce false negatives. The results of evaluating precision, recall, F-score, and number of samples for each behavior action class is listed in Table 5.","divisions":[{"label":"title","span":{"begin":0,"end":41}},{"label":"p","span":{"begin":42,"end":877}},{"label":"p","span":{"begin":878,"end":1745}},{"label":"p","span":{"begin":1746,"end":2337}}],"tracks":[{"project":"LitCovid-PubTator","denotations":[{"id":"387","span":{"begin":113,"end":118},"obj":"Species"},{"id":"388","span":{"begin":272,"end":280},"obj":"Disease"},{"id":"389","span":{"begin":653,"end":661},"obj":"Disease"},{"id":"390","span":{"begin":759,"end":767},"obj":"Disease"},{"id":"391","span":{"begin":817,"end":825},"obj":"Disease"},{"id":"395","span":{"begin":1309,"end":1315},"obj":"Species"},{"id":"396","span":{"begin":1430,"end":1436},"obj":"Species"},{"id":"397","span":{"begin":903,"end":911},"obj":"Disease"},{"id":"399","span":{"begin":2101,"end":2109},"obj":"Disease"},{"id":"403","span":{"begin":2480,"end":2488},"obj":"Disease"},{"id":"404","span":{"begin":2810,"end":2818},"obj":"Disease"},{"id":"405","span":{"begin":2890,"end":2898},"obj":"Disease"}],"attributes":[{"id":"A387","pred":"tao:has_database_id","subj":"387","obj":"Tax:9606"},{"id":"A388","pred":"tao:has_database_id","subj":"388","obj":"MESH:D003371"},{"id":"A389","pred":"tao:has_database_id","subj":"389","obj":"MESH:C000657245"},{"id":"A390","pred":"tao:has_database_id","subj":"390","obj":"MESH:D003371"},{"id":"A391","pred":"tao:has_database_id","subj":"391","obj":"MESH:C000657245"},{"id":"A395","pred":"tao:has_database_id","subj":"395","obj":"Tax:9606"},{"id":"A396","pred":"tao:has_database_id","subj":"396","obj":"Tax:9606"},{"id":"A397","pred":"tao:has_database_id","subj":"397","obj":"MESH:D003371"},{"id":"A399","pred":"tao:has_database_id","subj":"399","obj":"MESH:C000657245"},{"id":"A403","pred":"tao:has_database_id","subj":"403","obj":"MESH:D003371"},{"id":"A404","pred":"tao:has_database_id","subj":"404","obj":"MESH:D003371"},{"id":"A405","pred":"tao:has_database_id","subj":"405","obj":"MESH:D003371"},{"subj":"387","pred":"source","obj":"LitCovid-PubTator"},{"subj":"388","pred":"source","obj":"LitCovid-PubTator"},{"subj":"389","pred":"source","obj":"LitCovid-PubTator"},{"subj":"390","pred":"source","obj":"LitCovid-PubTator"},{"subj":"391","pred":"source","obj":"LitCovid-PubTator"},{"subj":"395","pred":"source","obj":"LitCovid-PubTator"},{"subj":"396","pred":"source","obj":"LitCovid-PubTator"},{"subj":"397","pred":"source","obj":"LitCovid-PubTator"},{"subj":"399","pred":"source","obj":"LitCovid-PubTator"},{"subj":"403","pred":"source","obj":"LitCovid-PubTator"},{"subj":"404","pred":"source","obj":"LitCovid-PubTator"},{"subj":"405","pred":"source","obj":"LitCovid-PubTator"}],"namespaces":[{"prefix":"Tax","uri":"https://www.ncbi.nlm.nih.gov/taxonomy/"},{"prefix":"MESH","uri":"https://id.nlm.nih.gov/mesh/"},{"prefix":"Gene","uri":"https://www.ncbi.nlm.nih.gov/gene/"},{"prefix":"CVCL","uri":"https://web.expasy.org/cellosaurus/CVCL_"}]},{"project":"LitCovid-PD-HP","denotations":[{"id":"T5","span":{"begin":272,"end":280},"obj":"Phenotype"},{"id":"T6","span":{"begin":759,"end":767},"obj":"Phenotype"},{"id":"T7","span":{"begin":903,"end":911},"obj":"Phenotype"},{"id":"T8","span":{"begin":2480,"end":2488},"obj":"Phenotype"},{"id":"T9","span":{"begin":2810,"end":2818},"obj":"Phenotype"},{"id":"T10","span":{"begin":2890,"end":2898},"obj":"Phenotype"}],"attributes":[{"id":"A5","pred":"hp_id","subj":"T5","obj":"http://purl.obolibrary.org/obo/HP_0012735"},{"id":"A6","pred":"hp_id","subj":"T6","obj":"http://purl.obolibrary.org/obo/HP_0012735"},{"id":"A7","pred":"hp_id","subj":"T7","obj":"http://purl.obolibrary.org/obo/HP_0012735"},{"id":"A8","pred":"hp_id","subj":"T8","obj":"http://purl.obolibrary.org/obo/HP_0012735"},{"id":"A9","pred":"hp_id","subj":"T9","obj":"http://purl.obolibrary.org/obo/HP_0012735"},{"id":"A10","pred":"hp_id","subj":"T10","obj":"http://purl.obolibrary.org/obo/HP_0012735"},{"subj":"T5","pred":"source","obj":"LitCovid-PD-HP"},{"subj":"T6","pred":"source","obj":"LitCovid-PD-HP"},{"subj":"T7","pred":"source","obj":"LitCovid-PD-HP"},{"subj":"T8","pred":"source","obj":"LitCovid-PD-HP"},{"subj":"T9","pred":"source","obj":"LitCovid-PD-HP"},{"subj":"T10","pred":"source","obj":"LitCovid-PD-HP"}]},{"project":"LitCovid-sentences","denotations":[{"id":"T376","span":{"begin":0,"end":4},"obj":"Sentence"},{"id":"T377","span":{"begin":5,"end":41},"obj":"Sentence"},{"id":"T378","span":{"begin":42,"end":112},"obj":"Sentence"},{"id":"T379","span":{"begin":113,"end":226},"obj":"Sentence"},{"id":"T380","span":{"begin":227,"end":477},"obj":"Sentence"},{"id":"T381","span":{"begin":478,"end":600},"obj":"Sentence"},{"id":"T382","span":{"begin":601,"end":682},"obj":"Sentence"},{"id":"T383","span":{"begin":683,"end":769},"obj":"Sentence"},{"id":"T384","span":{"begin":770,"end":877},"obj":"Sentence"},{"id":"T385","span":{"begin":878,"end":1088},"obj":"Sentence"},{"id":"T386","span":{"begin":1089,"end":1167},"obj":"Sentence"},{"id":"T387","span":{"begin":1168,"end":1227},"obj":"Sentence"},{"id":"T388","span":{"begin":1228,"end":1271},"obj":"Sentence"},{"id":"T389","span":{"begin":1272,"end":1464},"obj":"Sentence"},{"id":"T390","span":{"begin":1465,"end":1572},"obj":"Sentence"},{"id":"T391","span":{"begin":1573,"end":1666},"obj":"Sentence"},{"id":"T392","span":{"begin":1667,"end":1745},"obj":"Sentence"},{"id":"T393","span":{"begin":1746,"end":1827},"obj":"Sentence"},{"id":"T394","span":{"begin":1828,"end":1945},"obj":"Sentence"},{"id":"T395","span":{"begin":1946,"end":2131},"obj":"Sentence"},{"id":"T396","span":{"begin":2132,"end":2337},"obj":"Sentence"},{"id":"T397","span":{"begin":2338,"end":2543},"obj":"Sentence"},{"id":"T398","span":{"begin":2544,"end":2670},"obj":"Sentence"},{"id":"T399","span":{"begin":2671,"end":2760},"obj":"Sentence"},{"id":"T400","span":{"begin":2761,"end":2947},"obj":"Sentence"},{"id":"T401","span":{"begin":2948,"end":3026},"obj":"Sentence"},{"id":"T402","span":{"begin":3027,"end":3112},"obj":"Sentence"},{"id":"T403","span":{"begin":3113,"end":3217},"obj":"Sentence"},{"id":"T404","span":{"begin":3218,"end":3335},"obj":"Sentence"},{"id":"T405","span":{"begin":3336,"end":3464},"obj":"Sentence"}],"namespaces":[{"prefix":"_base","uri":"http://pubannotation.org/ontology/tao.owl#"}],"attributes":[{"subj":"T376","pred":"source","obj":"LitCovid-sentences"},{"subj":"T377","pred":"source","obj":"LitCovid-sentences"},{"subj":"T378","pred":"source","obj":"LitCovid-sentences"},{"subj":"T379","pred":"source","obj":"LitCovid-sentences"},{"subj":"T380","pred":"source","obj":"LitCovid-sentences"},{"subj":"T381","pred":"source","obj":"LitCovid-sentences"},{"subj":"T382","pred":"source","obj":"LitCovid-sentences"},{"subj":"T383","pred":"source","obj":"LitCovid-sentences"},{"subj":"T384","pred":"source","obj":"LitCovid-sentences"},{"subj":"T385","pred":"source","obj":"LitCovid-sentences"},{"subj":"T386","pred":"source","obj":"LitCovid-sentences"},{"subj":"T387","pred":"source","obj":"LitCovid-sentences"},{"subj":"T388","pred":"source","obj":"LitCovid-sentences"},{"subj":"T389","pred":"source","obj":"LitCovid-sentences"},{"subj":"T390","pred":"source","obj":"LitCovid-sentences"},{"subj":"T391","pred":"source","obj":"LitCovid-sentences"},{"subj":"T392","pred":"source","obj":"LitCovid-sentences"},{"subj":"T393","pred":"source","obj":"LitCovid-sentences"},{"subj":"T394","pred":"source","obj":"LitCovid-sentences"},{"subj":"T395","pred":"source","obj":"LitCovid-sentences"},{"subj":"T396","pred":"source","obj":"LitCovid-sentences"},{"subj":"T397","pred":"source","obj":"LitCovid-sentences"},{"subj":"T398","pred":"source","obj":"LitCovid-sentences"},{"subj":"T399","pred":"source","obj":"LitCovid-sentences"},{"subj":"T400","pred":"source","obj":"LitCovid-sentences"},{"subj":"T401","pred":"source","obj":"LitCovid-sentences"},{"subj":"T402","pred":"source","obj":"LitCovid-sentences"},{"subj":"T403","pred":"source","obj":"LitCovid-sentences"},{"subj":"T404","pred":"source","obj":"LitCovid-sentences"},{"subj":"T405","pred":"source","obj":"LitCovid-sentences"}]}],"config":{"attribute types":[{"pred":"source","value type":"selection","values":[{"id":"LitCovid-PubTator","color":"#93ec9e","default":true},{"id":"LitCovid-PD-HP","color":"#ec93a2"},{"id":"LitCovid-sentences","color":"#93bcec"}]}]}}