TextAnalyzer Pipeline

Python module for using NLP bias analysis models.

1. Import

# pip install the-fairly-project
from fairly import TextAnalyzer

2. Initialize the Module

text_pipeline = TextAnalyzer(
    bias="ternary", # defaults to None
    classes=True, # defaults to False
    top_k_classes=3, # defaults to 3
    ner="gus" 
    )

Customize your pipeline

Args	Options
bias	Classify bias at the sentence level (sequence classification): None: Default (no bias sequence classification). "ternary": Uses Unbias (e.g. Neutral, Slightly Biased, Highly Biased). More info. "binary": Not implemented yet. (e.g. Fair, Biased) More info. When set to ternary this adds two fields to the "text" dictionary in the return dictionary: label (as shown above) and score (0-1).
classes	Classify the types of bias a sentence contains: False: Default (no bias aspects classification). True: Uses fairlyAspects (11 classes). More info. When set to True, this adds one field to the "text" dictionary in the return dictionary: aspects(which contains top_k_classes in [CLASS]: [SCORE] format).
top_k_classes	Number of classes returned in the aspects dict. Int: 1 to 11 (defaults to 3). Only relevant when classes is set to True.
ner	Run named-entity recognition on the text sequence. None: Default (no token classification) "gus": Uses GUS-Net (e.g. GEN, UNFAIR, STEREO). More info. When in use, it appends a new "ner" dictionary to the return dictionary.

3. Run Bias Analysis

result = text_pipeline.analyze("Data scientists are so smart")

4. Example Output

{
'text': {
    'text': 'Tall people are so clumsy.', 
    'label': 'Slightly Biased', 
    'score': 0.6829080581665039, 
    'aspects': {
      'physical': 0.9650779366493225, 
      'gender': 0.024978743866086006, 
      'socioeconomic': 0.023334791883826256
    }
}, 
'ner': [
    {'token': 'tall', 'labels': ['B-STEREO', 'B-GEN', 'B-UNFAIR']}, 
    {'token': 'people', 'labels': ['I-STEREO', 'I-GEN', 'I-UNFAIR']}, 
    {'token': 'are', 'labels': ['I-STEREO']}, 
    {'token': 'so', 'labels': ['I-STEREO']}, 
    {'token': 'clumsy', 'labels': ['I-STEREO', 'B-UNFAIR', 'I-UNFAIR']},
    {'token': '.', 'labels': ['I-STEREO', 'I-UNFAIR']}
    ]
}