SHARED TASK @

CONSTRAINT 2021

First Workshop on ​Combating ​On​line Ho​st​ile Posts in ​Regional L​anguages dur​ing Emerge​ncy Si​tuation

Collocated with AAAI 2021

NEWS

ABOUT THE SHARED TASK


  • Tasks- The CONSTRAINT-2020 shared task on the hostile post detection invites participation in two subtasks:

    • COVID19 Fake News Detection in English - This subtask focuses on the detection of COVID19-related fake news in English. The sources of data are various social-media platforms such as Twitter, Facebook, Instagram, etc. Given a social media post, the objective of the shared task is to classify it into either fake or real news. For example, the following two posts belong to fake and real categories, respectively.

      If you take Crocin thrice a day you are safe. Fake
      Wearing mask can protect you from the virus Real

      English Dataset: https://competitions.codalab.org/competitions/26655


    • Hostile Post Detection in Hindi -This subtask focuses on a variety of hostile posts in Hindi Devanagari script collected from Twitter and Facebook. The set of valid categories are fake news, hate speech, offensive, defamation, and non-hostile posts. It is a multi-label multi-class classification problem where each post can belong to one or more of these hostile classes. However, the non-hostile posts cannot be grouped with any other class. The evaluation of this subtask will be two-dimensional as follows:
      1. Coarse-grained evaluation: It is a binary evaluation of hostile vs non-hostile posts.
      2. Fine-grained evaluation: It is a fine-grained evaluation of the hostile classes.

      Definitions of the class labels:
      • Fake News: A claim or information that is verified to be not true.
      • Hate Speech: A post targeting a specific group of people based on their ethnicity, religious beliefs, geographical belonging, race, etc., with malicious intentions of spreading hate or encouraging violence.
      • Offensive: A post containing profanity, impolite, rude, or vulgar language to insult a targeted individual or group.
      • Defamation: A mis-information regarding an individual or group.
      • Non-hostile: A post without any hostility.

      Examples

      ये देखो इस्लाम क्या क्या सिखाता है जिहाद से लेकर आतंकवादी और दंगों से लेकर चोरी बुर्खे की आड़ में चद्दर चुराती महिलाएं {hate}
      मोहतरमा JNU की 43 साल की छात्रा हैं , और कमाल की बात है कि उनकी बेटी मोना भी JNU में ही पड़ती है {Fake, Defamation}
      जब इन दलितों को (सभी नहीं) हिन्दू धर्म और हिन्दू देवी देवताओं से इतनी नफरत भारी हुई है तो धूर्त कहीं के अपना नाम हिन्दुओं के जैसे ही क्यों रखते हैं। किसने रोका है कुछ भी बन से, बन जाओ मुस्लिम, ईसाई और जो मन करे। इस धूर्त की हिम्मत नहीं कि किसी दूसरे धर्म के बारे ऐसा बोल दे । {Hate, Offensive}
      डॉक्टर कफ़ील ख़ान को हाईकोर्ट से मिली ज़मानत https://t.co/DH5WE370XT {Non-hostile}

    • Evaluation Metric: The official evaluation metric for the shared task is weighted-average F1 score.
      Hindi Dataset: https://competitions.codalab.org/competitions/26654
      Submission: Each team should submit a csv file in the following format for the final evaluation:
      <unique_id, {labels}>
      In case of multiple submissions by a team, we shall consider the last submission prior to the deadline for the final evaluation. No exceptions shall be made.
      System description paper: All team/participants will be invited to submit their models as short papers to be included in the proceedings. Based on the reviewers' comments, we will decide which papers to be accepted.
      Submission details: TBA

SHARED TASK

  • Important Dates:
    • October 1, 2020: Release of the training set
    • December 1, 2020: Release of the test set
    • December 10, 2020: Deadline for submitting the final results
    • December 12, 2020: Announcement of the results
    • December 30, 2020: System paper submission deadline (All teams are invited to submit a paper)

  • System Paper Submission Instructions:
    • TBD

CONTACT US

Email
  tanmoy@iiitd.ac.in
  parthprasad.p17@iiits.in

Subscribe to our mailing list      Click Here

Follow Us