Effectiveness of Model and Data Scale Contrastive Learning in Non-IID Federated Learning

Girum Fitihamlak Ejigu; Ye Lin Tun; Apurba Adhikary; Sun Moo Kang; Choong Seon Hong

doi:10.1109/ATC58710.2023.10318526

Effectiveness of Model and Data Scale Contrastive Learning in Non-IID Federated Learning

Girum Fitihamlak Ejigu, Ye Lin Tun, Apurba Adhikary, Sun Moo Kang, Choong Seon Hong

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

1 Citation (Scopus)

Abstract

All involved clients are guaranteed data privacy in a collaborative machine learning environment via Federated Learning. The lack of generalization in local client models brought on by data heterogeneity, however, is one of Federated Learning 's major challenges that lead to slow model convergence and communication latency. In this study, we use contrastive learning techniques that have been shown effective in both centralized and federated learning environments. For local models to have a stronger capacity for generalization, we suggest adopting contrastive loss at the model and data scales. Utilizing the CIFAR10 and CIFAR-100 datasets, we assess and contrast our suggested strategy with other industry standard approaches.

Original language	English
Title of host publication	Proceedings - 16th International Conference on Advanced Technologies for Communications, ATC 2023
Editors	Tran The Son
Publisher	IEEE Computer Society
Pages	301-304
Number of pages	4
ISBN (Electronic)	9798350301328
DOIs	https://doi.org/10.1109/ATC58710.2023.10318526
Publication status	Published - 2023
Event	16th International Conference on Advanced Technologies for Communications, ATC 2023 - Da Nang, Viet Nam Duration: 19 Oct 2023 → 21 Oct 2023

Publication series

Name	International Conference on Advanced Technologies for Communications
ISSN (Print)	2162-1039
ISSN (Electronic)	2162-1020

Conference

Conference	16th International Conference on Advanced Technologies for Communications, ATC 2023
Country/Territory	Viet Nam
City	Da Nang
Period	19/10/23 → 21/10/23

Bibliographical note

Publisher Copyright:
© 2023 IEEE.

Keywords

Federated learning
contrastive learning
federated optimization
non-IID

Access to Document

10.1109/ATC58710.2023.10318526

Cite this

Ejigu, G. F., Tun, Y. L., Adhikary, A., Kang, S. M., & Hong, C. S. (2023). Effectiveness of Model and Data Scale Contrastive Learning in Non-IID Federated Learning. In T. T. Son (Ed.), Proceedings - 16th International Conference on Advanced Technologies for Communications, ATC 2023 (pp. 301-304). (International Conference on Advanced Technologies for Communications). IEEE Computer Society. https://doi.org/10.1109/ATC58710.2023.10318526

Ejigu, Girum Fitihamlak ; Tun, Ye Lin ; Adhikary, Apurba et al. / Effectiveness of Model and Data Scale Contrastive Learning in Non-IID Federated Learning. Proceedings - 16th International Conference on Advanced Technologies for Communications, ATC 2023. editor / Tran The Son. IEEE Computer Society, 2023. pp. 301-304 (International Conference on Advanced Technologies for Communications).

@inproceedings{7c2244f0ceb6461197ffd6229f0a23cd,

title = "Effectiveness of Model and Data Scale Contrastive Learning in Non-IID Federated Learning",

abstract = "All involved clients are guaranteed data privacy in a collaborative machine learning environment via Federated Learning. The lack of generalization in local client models brought on by data heterogeneity, however, is one of Federated Learning 's major challenges that lead to slow model convergence and communication latency. In this study, we use contrastive learning techniques that have been shown effective in both centralized and federated learning environments. For local models to have a stronger capacity for generalization, we suggest adopting contrastive loss at the model and data scales. Utilizing the CIFAR10 and CIFAR-100 datasets, we assess and contrast our suggested strategy with other industry standard approaches.",

keywords = "Federated learning, contrastive learning, federated optimization, non-IID",

author = "Ejigu, {Girum Fitihamlak} and Tun, {Ye Lin} and Apurba Adhikary and Kang, {Sun Moo} and Hong, {Choong Seon}",

note = "Publisher Copyright: {\textcopyright} 2023 IEEE.; 16th International Conference on Advanced Technologies for Communications, ATC 2023 ; Conference date: 19-10-2023 Through 21-10-2023",

year = "2023",

doi = "10.1109/ATC58710.2023.10318526",

language = "English",

series = "International Conference on Advanced Technologies for Communications",

publisher = "IEEE Computer Society",

pages = "301--304",

editor = "Son, {Tran The}",

booktitle = "Proceedings - 16th International Conference on Advanced Technologies for Communications, ATC 2023",

address = "United States",

}

Ejigu, GF, Tun, YL, Adhikary, A, Kang, SM & Hong, CS 2023, Effectiveness of Model and Data Scale Contrastive Learning in Non-IID Federated Learning. in TT Son (ed.), Proceedings - 16th International Conference on Advanced Technologies for Communications, ATC 2023. International Conference on Advanced Technologies for Communications, IEEE Computer Society, pp. 301-304, 16th International Conference on Advanced Technologies for Communications, ATC 2023, Da Nang, Viet Nam, 19/10/23. https://doi.org/10.1109/ATC58710.2023.10318526

Effectiveness of Model and Data Scale Contrastive Learning in Non-IID Federated Learning. / Ejigu, Girum Fitihamlak; Tun, Ye Lin; Adhikary, Apurba et al.
Proceedings - 16th International Conference on Advanced Technologies for Communications, ATC 2023. ed. / Tran The Son. IEEE Computer Society, 2023. p. 301-304 (International Conference on Advanced Technologies for Communications).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Effectiveness of Model and Data Scale Contrastive Learning in Non-IID Federated Learning

AU - Ejigu, Girum Fitihamlak

AU - Tun, Ye Lin

AU - Adhikary, Apurba

AU - Kang, Sun Moo

AU - Hong, Choong Seon

PY - 2023

Y1 - 2023

N2 - All involved clients are guaranteed data privacy in a collaborative machine learning environment via Federated Learning. The lack of generalization in local client models brought on by data heterogeneity, however, is one of Federated Learning 's major challenges that lead to slow model convergence and communication latency. In this study, we use contrastive learning techniques that have been shown effective in both centralized and federated learning environments. For local models to have a stronger capacity for generalization, we suggest adopting contrastive loss at the model and data scales. Utilizing the CIFAR10 and CIFAR-100 datasets, we assess and contrast our suggested strategy with other industry standard approaches.

AB - All involved clients are guaranteed data privacy in a collaborative machine learning environment via Federated Learning. The lack of generalization in local client models brought on by data heterogeneity, however, is one of Federated Learning 's major challenges that lead to slow model convergence and communication latency. In this study, we use contrastive learning techniques that have been shown effective in both centralized and federated learning environments. For local models to have a stronger capacity for generalization, we suggest adopting contrastive loss at the model and data scales. Utilizing the CIFAR10 and CIFAR-100 datasets, we assess and contrast our suggested strategy with other industry standard approaches.

KW - Federated learning

KW - contrastive learning

KW - federated optimization

KW - non-IID

UR - http://www.scopus.com/inward/record.url?scp=85179136459&partnerID=8YFLogxK

U2 - 10.1109/ATC58710.2023.10318526

DO - 10.1109/ATC58710.2023.10318526

M3 - Conference contribution

AN - SCOPUS:85179136459

T3 - International Conference on Advanced Technologies for Communications

SP - 301

EP - 304

BT - Proceedings - 16th International Conference on Advanced Technologies for Communications, ATC 2023

A2 - Son, Tran The

PB - IEEE Computer Society

T2 - 16th International Conference on Advanced Technologies for Communications, ATC 2023

Y2 - 19 October 2023 through 21 October 2023

ER -

Ejigu GF, Tun YL, Adhikary A, Kang SM, Hong CS. Effectiveness of Model and Data Scale Contrastive Learning in Non-IID Federated Learning. In Son TT, editor, Proceedings - 16th International Conference on Advanced Technologies for Communications, ATC 2023. IEEE Computer Society. 2023. p. 301-304. (International Conference on Advanced Technologies for Communications). doi: 10.1109/ATC58710.2023.10318526

Effectiveness of Model and Data Scale Contrastive Learning in Non-IID Federated Learning

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this