A checklist for developing fair models
With the preceding information, we can create a short checklist that can be used when creating fair models. Each issue comes with several sub-issues.
What is the goal of the model developers?
- Is fairness an explicit goal?
- Is the model evaluation metric chosen to reflect the fairness of the model?
- How do model developers get promoted and rewarded?
- How does the model influence business results?
- Would the model discriminate against the developer's demographic?
- How diverse is the development team?
- Who is responsible when things go wrong?
- How was the data collected?
- Are there statistical misrepresentations in the sample?
- Are sample sizes for minorities adequate?
- Are sensitive variables included?
- Can sensitive variables be inferred from the data?
- Are there interactions between features that might only affect subgroups?
- What are the error rates for different subgroups?
- What is the error rate of a simple, rule-based alternative?
- How do the errors in the model lead to different outcomes?
How is feedback incorporated?
- Is there an appeals/reporting process?
- Can mistakes be attributed back to a model?
- Do model developers get insight into what happens with their model's predictions?
- Can the model be audited?
- Is the model open source?
- Do people know which features are used to make predictions about them?
Can the model be interpreted?
- Is a model interpretation, for example, individual results, in place?
- Can the interpretation be understood by those it matters to?
- Can findings from the interpretation lead to changes in the model?
What happens to models after deployment?
- Is there a central repository to keep track of all the models deployed?
- Are input assumptions checked continuously?
- Are accuracy and fairness metrics monitored continuously?
..................Content has been hidden....................
You can't read the all page of ebook, please click
here login for view all page.