Computer Science ›› 2024, Vol. 51 ›› Issue (7): 405-412.doi: 10.11896/jsjkx.230500012

• Information Security • Previous Articles     Next Articles

Lagrangian Dual-based Privacy Protection and Fairness Constrained Method for Few-shot Learning

WANG Jinghong1,2,3, TIAN Changshen1,2,3, LI Haokang4, WANG Wei1,3   

  1. 1 College of Computer and Cyber Security,Hebei Normal University,Shijiazhuang 050024,China
    2 Hebei Key Laboratory of Network and Information Security,Hebei Normal University,Shijiazhuang 050024,China
    3 Hebei Provincial Engineering Research Center for Supply Chain Big Data Analytics & Security,Hebei Normal University,Shijiazhuang 050024,China
    4 Artificial Intelligence and Big Data College,Hebei University of Engineering Science,Shijiazhuang 050020,China
  • Received:2023-04-30 Revised:2023-08-29 Online:2024-07-15 Published:2024-07-10
  • About author:WANG Jinghong,born in 1967,Ph.D,professor,academic advisor,is a member of CCF(No.58341S).Her main research interests include machine lear-ning,data mining,and artificial intelligence.
    WANG Wei,born in 1982,Ph.D,asso-ciate professor,academic advisor,is a member of CCF(No.51382M).His main research interests include machine learning,knowledge representation,and virtual simulation.
  • Supported by:
    Natural Science Foundation of Hebei,China(F2021205014),Science and Technology Project of Hebei Education Department(ZD2022139),Central Guidance on Local Science and Technology Development Fund of Hebei Province(226Z1808G),Project Funded by the Introduction of Overseas Students in Hebei Province(C20200340) and Science Foundation of Hebei Normal University(L2022B22).

Abstract: Few-shot learning aims to use a small amount of data for training and significantly improve model performance,and is an important approach to address privacy and fairness issues of sensitive data in neural network models.In few-shot learning,there is a risk of privacy and fairness issues in training neural network models due to the fact that small sample datasets often contain certain sensitive data,and that such sensitive data may be discriminatory.In addition,in many domains,data is difficult or impossible to access for reasons such as privacy or security.Also,in differential privacy models,the introduction of noise not only leads to a reduction in model utility,but also causes an imbalance in model fairness.To address these challenges,this paper proposes a sample-level adaptive privacy filtering algorithm based on the Rényi differential privacy filter to exploit Rényi differential privacy to achieve a more accurate calculation of privacy loss.Furthermore,it proposes a Lagrangian dual-based privacy and fairness constraint algorithm,which adds the differential privacy constraint and the fairness constraint to the objective function by introducing a Lagrangian method,and introduces a Lagrangian multiplier to balance these constraints.The Lagrangian multiplier method is used to transform the objective function into a pairwise problem,thus optimising both privacy and fairness,and achieving a balance between privacy and fairness through the Lagrangian function.It is shown that the proposed method improves the performance of the model while ensuring privacy and fairness of the model.

Key words: Few-shot learning, Privacy and fairness, Rényi differential privacy, Fairness constraint, Lagrangian dual

CLC Number: 

  • TP309.3
[1]CHANG H,SHOKRI R.On the privacy risks of algorithmicfairness[C]//2021 IEEE European Symposium on Security and Privacy(EuroS&P).IEEE,2021:292-303.
[2]FARRAND T,MIRESHGHALLAH F,SINGH S,et al.Neither private nor fair:Impact of data imbalance on utility and fairness in differential privacy[C]//Proceedings of the 2020 Workshop on Privacy-preserving Machine Learning in Practice.Association for Computing Machinery,2020:15-19.
[3]LIN Y,BAO L Y,LI Z M H,et al.Differential privacy protection over deep learning:An investigation of its impacted factors[J].Computers & Security,2020,99:102061.
[4]GARCIA V,BRUNA J.Few-shot learning with graph neuralnetworks[J].arXiv:1711.04043,2017.
[5]MIRONOV I.Rényi differential privacy[C]//2017 IEEE 30th Computer Security Foundations Symposium.IEEE,2017:263-275.
[6]DWORK C.Differential privacy [C]//Proceedings of the 33rd International Colloquium on Automata,Languages and Programming(ICALP),Part II.2006:1-12.
[7]ABADI M,CHU A,GOODFELLOW I,et al.Deep learning withdifferential privacy[C]//Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security.ACM,2016:308-318.
[8]FELDMAN V,ZRNIC T.Individual privacy accounting via aRényi filter[C]//Advances in neural information processing systems.New York:Curran Associates,Inc.2021:28080-28091.
[9]DU M N,YANG F,ZOU N,et al.Fairness in deep learning:a computational perspective[J].IEEE Intelligent Systems,2021,4(36):25-34.
[10]ZHANG T,ZHU T,LI J,et al.Fairness in semi-supervisedlearning:Unlabeled data help to reduce discrimination[J].IEEE Transactions on Knowledge and Data Engineering,2020,34(4):1763-1774.
[11]ZAFAR M B,VALERA I,ROGRIGUEZ M G,et al.Fairnessconstraints:Mechanisms for fair classification[C]//Artificial Intelligence and Statistics.PMLR,2017:962-970.
[12]LIANG Y,CHEN C,TIAN T,et al.Joint Adversarial Learning for Cross-domain Fair Classification[J].arXiv:2206.03656,2022.
[13]DU M,YANG F,ZOU N,et al.Fairness in deep learning:A computational perspective[J].IEEE Intelligent Systems,2020,36(4):25-34.
[14]PENG Y C,QIN X I,ZHANG L G,et al.Survey on Few-shot Learning Algorithms for Image Classification[J].Computer Science,2022,49(5):1-9.
[15]LU J Y,LING X H,LIU Q,et al.Meta-reinforcement Learning Algorithm Based on Automating Policy Entropy[J].Computer Science,2021,48(6):168-174.
[16]VINYALS O,BLUNDELL C,LILLICRAP T,et al.Ma-tching networks for one shot learning[J].Advances in Neural Information Processing Systems,2016,29:3630-3638.
[17]TRIANTAFILLOU E,ZHU T,DUMOULIN V,et al.Meta-dataset:A dataset of datasets for learning to learn from few examples[J].arXiv:1903.03096,2019.
[18]TSIMPOUKELLI M,MENICK J L,CABI S,et al.Multimodal few-shot learning with frozen language models[J].Advances in Neural Information Processing Systems,2021,34:200-212.
[19]SNELL J,SWERSKY K,ZEMEL R.Prototypical networks for few-shot learning[J].Advances in Neural Information Proces-sing Systems,2017,30:4077-4087.
[20]RAVI S,LAROCHELLE H.Optimization as a model for few-shot learning[C]//International Conference on Learning Representations.2017.
[21]JAMAL M A,QI G J.Task-agnostic meta-learning for few-shot learning[C]//Conference on Computer Vision and Pattern Recognition.IEEE,2019:11719-11727.
[22]FINN C,ABBEEL P,LEVINE S.Model-agnostic meta-learning for fast adaptation of deep networks[C]//International Confe-rence on Machine Learning.PMLR,2017:1126-1135.
[23]ABBAS M,XIAO Q,CHEN L,et al.Sharp-MAML:Sharpness-Aware Model-Agnostic Meta Learning[C]//International Conference on Machine Learning.PMLR,2022:10-32.
[24]LI J,KHODAK M,CALDAS S,et al.Differentially private meta-learning[J].arXiv:1909.05830,2019.
[25]SLACK D,FRIEDLER S A,GIVENTAL E.Fairness warnings and Fair-MAML:learning fairly with minimal data[C]//Proceedings of the 2020 Conference on Fairness,Accountability,and Transparency.ACM,2020:200-209.
[26]ZHAO C,CHEN F,WANG Z,et al.A primal-dual subgradient approach for fair meta learning[C]//2020 IEEE International Conference on Data Mining(ICDM).IEEE,2020:821-830.
[27]JAGIELSKI M,KEARNS M,MAO J M,et al.Differenially private fair learning[C]//International Conference on Machine Learning.PMLR,2019:3000-3008.
[28]FIORETTO F,MAK T W K,VAN HENTENRYCK P.Predicting ac optimal power flows:Combining deep learning and lagrangian dual methods[C]//Proceedings of the AAAI Confe-rence on Artificial Intelligence.2020:630-637.
[29]TRAN C,FIORETTO F,VAN HENTENRYCK P.Differen-tially private and fair deep learning:A lagrangian dual approach[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2021,35(11):9932-9939.
[30]DU M,MUKHERJEE S,WANG G,et al.Fairness via representation neutralization[J].Advances in Neural Information Processing Systems,2021,34:12091-12103.
[31]BECHAVOD Y,LIGETT K.Penalizing unfairness in binaryclassification[J].arXiv:1707.00044,2017.
[32]XIE Y,WANG H,YU B,et al.Secure collaborative few-shot learning[J].Knowledge-Based Systems,2020,203:106157.
[1] YANG Xuhua, ZHANG Lian, YE Lei. Adaptive Context Matching Network for Few-shot Knowledge Graph Completion [J]. Computer Science, 2024, 51(5): 223-231.
[2] ZENG Wu, MAO Guojun. Few-shot Learning Method Based on Multi-graph Feature Aggregation [J]. Computer Science, 2023, 50(6A): 220400029-10.
[3] HUA Jie, LIU Xueliang, ZHAO Ye. Few-shot Object Detection Based on Feature Fusion [J]. Computer Science, 2023, 50(2): 209-213.
[4] PENG Yun-cong, QIN Xiao-lin, ZHANG Li-ge, GU Yong-xiang. Survey on Few-shot Learning Algorithms for Image Classification [J]. Computer Science, 2022, 49(5): 1-9.
[5] GUO Jun-cheng, WAN Gang, HU Xin-jie, WANG Shuai, YAN Fa-bao. Study on Solar Radio Burst Event Detection Based on Transfer Learning [J]. Computer Science, 2022, 49(11A): 210900198-7.
[6] FANG Zhong-li, WANG Zhe, CHI Zi-qiu. Dual-stream Reconstruction Network for Multi-label and Few-shot Learning [J]. Computer Science, 2022, 49(1): 212-218.
[7] WANG Hang, CHEN Xiao, TIAN Sheng-zhao, CHEN Duan-bing. SAR Image Recognition Based on Few-shot Learning [J]. Computer Science, 2020, 47(5): 124-128.
[8] LV Yong-qiang,MIN Wei-qing,DUAN Hua,JIANG Shu-qiang. Few-shot Food Recognition Combining Triplet Convolutional Neural Network with Relation Network [J]. Computer Science, 2020, 47(1): 136-143.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!