This assignment aims to give students practical experience in using existing popular toolboxes for trustworthy AI research. The selected tools cover diverse aspects of trustworthy AI, including fairness, interpretability, adversarial robustness, domain generalization, and semi-supervised learning. By using these tools, the students will be better prepared for their future research.
Leverage the following open-source software (you are welcome to use your own) to investigate problems with one of existing AI systems (image classification, natural language processing, time series forecasting, finance, etc.). You should first get familiar with the tools you choose to use and then run experiments to clearly show their weakness. For example, in an image classification system, you can use adversarial robustness toolbox to attack it, then you can identify that this system is not secure since its accuracy can be dropped from xx% to xx% because of your attack.
| ID | Tool/Codebase | Area of Focus | URL |
|---|---|---|---|
| 1 | AIF360 | Fairness and Bias Detection | ‣ |
| 2 | InterpretML | Interpretability | ‣ |
| 3 | ART | Adversarial Robustness | ‣ |
| 4 | DomainBed | Domain Generalization | ‣ |
| 5 | USB | Semi-supervised Learning | ‣ |
| 6 | PromptBench | Large Language Models | ‣ |
| 7 | PersonalizedFL | Federated Learning | ‣ |
A Jupyter notebook containing all your code and running results, with a 4~6-page report (also in NeurIPS format) of your experiments.