{"id":576,"date":"2025-04-18T21:55:00","date_gmt":"2025-04-18T13:55:00","guid":{"rendered":"https:\/\/blogqian.com\/?p=576"},"modified":"2025-07-04T11:17:10","modified_gmt":"2025-07-04T03:17:10","slug":"ai_abstract_my_dierection","status":"publish","type":"post","link":"https:\/\/blogqian.com\/index.php\/2025\/04\/18\/ai_abstract_my_dierection\/","title":{"rendered":"\u4eba\u5de5\u667a\u80fd\u591a\u6a21\u6001 &#8211; \u7efc\u8ff0"},"content":{"rendered":"\n<h1 class=\"wp-block-heading\">\u5165\u95e8\u7efc\u8ff0<\/h1>\n\n\n\n<figure class=\"wp-block-image size-full\"><div class='fancybox-wrapper lazyload-container-unload' data-fancybox='post-images' href='https:\/\/blogqian.com\/wp-content\/uploads\/2025\/07\/\u622a\u5c4f2025-04-19-21.45.07-scaled.png'><img class=\"lazyload lazyload-style-8\" src=\"data:image\/svg+xml;base64,PCEtLUFyZ29uTG9hZGluZy0tPgo8c3ZnIHdpZHRoPSIxIiBoZWlnaHQ9IjEiIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8yMDAwL3N2ZyIgc3Ryb2tlPSIjZmZmZmZmMDAiPjxnPjwvZz4KPC9zdmc+\"  loading=\"lazy\" decoding=\"async\" width=\"2560\" height=\"725\" data-original=\"https:\/\/blogqian.com\/wp-content\/uploads\/2025\/07\/\u622a\u5c4f2025-04-19-21.45.07-scaled.png\" src=\"data:image\/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAAJcEhZcwAADsQAAA7EAZUrDhsAAAANSURBVBhXYzh8+PB\/AAffA0nNPuCLAAAAAElFTkSuQmCC\" alt=\"\" class=\"wp-image-577\"  sizes=\"auto, (max-width: 2560px) 100vw, 2560px\" \/><\/div><\/figure>\n\n\n\n<p>\u591a\u6a21\u6001\u662f\u4e00\u4e2a\u878d\u5408\u6027\u7684\u9886\u57df\uff0c\u5176\u672c\u8d28\u662f\u5904\u7406heterogeneous\u7684Data\uff0c\u5c06\u4e0d\u540c\u7684modalities\uff08e.g. Vision, Acoustic, Touch, Language)\u653e\u5230\u4e00\u8d77\u8fdb\u884c\u878d\u5408\u8bad\u7ec3\u3002\uff08\u4e0d\u540cModalities\u4f1a\u5728\u5143\u7d20\u79cd\u7c7b\uff0c\u5206\u5e03\uff0c\u6570\u636e\u7ed3\u6784\uff0c\u6db5\u76d6\u4fe1\u606f\u7684\u4e30\u5ea6\uff0c\u566a\u58f0\u7b49\u7ea7\uff0c\u4e0etask\u7684\u76f8\u5173\u5ea6\u7b49\u65b9\u9762\u6709\u533a\u522b<a href=\"#id1\">[1]<\/a>\uff09\u3002<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><div class='fancybox-wrapper lazyload-container-unload' data-fancybox='post-images' href='https:\/\/blogqian.com\/wp-content\/uploads\/2025\/07\/\u622a\u5c4f2025-04-20-01.12.33.png'><img class=\"lazyload lazyload-style-8\" src=\"data:image\/svg+xml;base64,PCEtLUFyZ29uTG9hZGluZy0tPgo8c3ZnIHdpZHRoPSIxIiBoZWlnaHQ9IjEiIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8yMDAwL3N2ZyIgc3Ryb2tlPSIjZmZmZmZmMDAiPjxnPjwvZz4KPC9zdmc+\"  loading=\"lazy\" decoding=\"async\" width=\"2076\" height=\"360\" data-original=\"https:\/\/blogqian.com\/wp-content\/uploads\/2025\/07\/\u622a\u5c4f2025-04-20-01.12.33.png\" src=\"data:image\/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAAJcEhZcwAADsQAAA7EAZUrDhsAAAANSURBVBhXYzh8+PB\/AAffA0nNPuCLAAAAAElFTkSuQmCC\" alt=\"\" class=\"wp-image-578\"  sizes=\"auto, (max-width: 2076px) 100vw, 2076px\" \/><\/div><\/figure>\n\n\n\n<p>\u6839\u636e\u6700\u7ecf\u5178\u7684\u7efc\u8ff0\u8bba\u6587<a href=\"#id2\">[2]<\/a>\uff0c\u878d\u5408\u8bad\u7ec3\u7684\u65b9\u5f0f\u53ef\u4ee5\u5206\u4e3a\u4ee5\u4e0b\u51e0\u79cd\u7c7b\u578b\uff1a<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Early Fusion\uff1a\u5728data\u8f93\u5165\u7684\u65f6\u5019\u5c31\u8fdb\u884c\u7ed3\u5408\uff08e.g.\u56fe\u50cf\u50cf\u7d20\u4fe1\u606f\u548c\u6587\u672c\u90fd\u6620\u5c04\u5230\u4e00\u4e2a\u7a7a\u95f4,\u5373\u5c06\u6587\u672c\u548c\u56fe\u50cf\u7279\u5f81\u76f4\u63a5\u62fc\u63a5\uff09<\/li>\n\n\n\n<li>Mid Fusion\uff1a\u5728\u4e2d\u95f4\u5c42\u8fdb\u884c\u4ea4\u4e92<a href=\"#id3\">[3]<\/a><a href=\"#id4\">[4]<\/a><a href=\"#id5\">[5]<\/a>\uff08e.g. \u5229\u7528Transformer\u673a\u5236\uff0c\u5728 Transformer \u7684\u4e2d\u95f4\u5c42\u8fdb\u884c cross-attention \u6216\u878d\u5408<a href=\"#id3\">[3]<\/a>\uff1bCLIP\u6a21\u578b\uff0c\u53cc\u5854\u7ed3\u6784\uff0c\u5bf9\u6bd4\u635f\u5931\u4f18\u5316\uff0c\u4f7f\u56fe\u6587\u4e24\u8005\u5206\u522b\u7684\u7f16\u7801\u5668\u5728\u4e00\u4e2a\u7a7a\u95f4\u91cc\u8fdb\u884c\u4f18\u5316\u5bf9\u9f50<a href=\"#id4\">[4]<\/a> )<\/li>\n\n\n\n<li>Late Fusion\uff1a\u4e0d\u540cmodality\u7684\u6a21\u578b\u72ec\u7acb\u7684\u8fdb\u884c\u8f93\u5165\uff0c\u8ba1\u7b97\uff0c\u548c\u5f97\u51fa\u7ed3\u679c\uff0c\u5728\u6700\u540e\u901a\u8fc7\u6295\u7968\u6216\u8005\u52a0\u6743\u5e73\u5747\u7b49\u65b9\u5f0f\u5f97\u5230\u7ed3\u679c\u3002<\/li>\n<\/ul>\n\n\n\n<p>\u5176\u4e2d\uff0c\u57282018\u5e74\u4ee5\u524d\uff0cEarly Fusion\u548cLate Fusion\u5360\u636e\u4e3b\u6d41\u5730\u4f4d\uff0c\u4f46\u662f\u5728\u6b64\u4e4b\u540eMid Fusion\uff08\u62e5\u62b1transformer\u673a\u5236\uff09\u9010\u6e10\u6210\u4e3a\u4e3b\u6d41\u65b9\u6cd5<a href=\"#id1\">[1]<\/a>\u3002<\/p>\n\n\n\n<p>\u540c\u65f6\u5462\uff0c\u8fd9\u7bc7\u6587\u7ae0<a href=\"#id1\">[1]<\/a>\u4e5f\u4ece\u5b8f\u89c2\u7684\u89d2\u5ea6\u8bb2\u89e3\u4e86\u76ee\u524d\u591a\u6a21\u6001\u6240\u9762\u4e34\u7684\u4e00\u4e9b\u6311\u6218\u548c\u65b9\u5411\uff1a<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Presentation\uff1a\u5373\u5bf9\u4e8e\u5143\u7d20\u7684\u8868\u793a\uff0c\u8fd9\u79cd\u8868\u793a\u53ef\u4ee5\u662f\u201c\u5c40\u90e8\u201d\u7684\uff0c\u4e5f\u53ef\u4ee5\u662f\u878d\u5408\u4e24\u4e2a\u5143\u7d20\u7684\u201c\u6574\u4f53\u201d\u8868\u793a\u3002 \u603b\u7684\u6765\u8bf4\u53ef\u5206\u4e3a\u4e09\u7c7b\uff1aFusion\uff08\u5373\u7528\u878d\u5408\u7684\u8868\u793a\u53bb\u8868\u793a\u4e0d\u540c\u7684modality\uff09\uff0cCoordination\uff08\u5373\u5bf9\u4e0d\u540c\u8868\u793a\u4e4b\u95f4\u8fdb\u884c\u5bf9\u9f50\uff09\uff0cFission\uff08\u5373\u521b\u9020\u4e00\u79cd\u65b0\u7684\u8868\u793a\uff0c\u53ef\u4ee5\u662f\u4e00\u79cd\u65b0\u7684modality\uff09<\/li>\n<\/ul>\n\n\n\n<figure class=\"wp-block-image size-full\"><div class='fancybox-wrapper lazyload-container-unload' data-fancybox='post-images' href='https:\/\/blogqian.com\/wp-content\/uploads\/2025\/07\/\u622a\u5c4f2025-04-20-01.51.21.png'><img class=\"lazyload lazyload-style-8\" src=\"data:image\/svg+xml;base64,PCEtLUFyZ29uTG9hZGluZy0tPgo8c3ZnIHdpZHRoPSIxIiBoZWlnaHQ9IjEiIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8yMDAwL3N2ZyIgc3Ryb2tlPSIjZmZmZmZmMDAiPjxnPjwvZz4KPC9zdmc+\"  loading=\"lazy\" decoding=\"async\" width=\"2076\" height=\"360\" data-original=\"https:\/\/blogqian.com\/wp-content\/uploads\/2025\/07\/\u622a\u5c4f2025-04-20-01.51.21.png\" src=\"data:image\/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAAJcEhZcwAADsQAAA7EAZUrDhsAAAANSURBVBhXYzh8+PB\/AAffA0nNPuCLAAAAAElFTkSuQmCC\" alt=\"\" class=\"wp-image-579\"  sizes=\"auto, (max-width: 2076px) 100vw, 2076px\" \/><\/div><\/figure>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Alignment\uff1a\u5373\u7814\u7a76\u4e0d\u540c\u7684modality\u662f\u5982\u4f55\u8fde\u63a5\u7684\uff08\u4f8b\u5982\u56fe\u6587\u8f6c\u6362\u4e2d\uff0c\u7279\u5b9a\u7684\u6587\u5b57\u548c\u7279\u5b9a\u7684\u56fe\u7247\u4e4b\u95f4\u7684\u8fde\u63a5\uff0c\u8fd9\u91cc\u4e0d\u96be\u770b\u51fa\uff0cAlignment\u4e0d\u662f\u4e00\u5bf9\u4e00\u5173\u7cfb\uff0c\u800c\u662f\u53ef\u80fd\u4e00\u5bf9\u591a\uff0c\u591a\u5bf9\u591a\uff0c\u6bd4\u5982\u4e00\u4e2a\u6587\u5b57\u53ef\u4ee5\u7531\u591a\u4e2a\u56fe\u50cf\u6765\u8868\u793a\uff09\u5728\u672c\u6587\u4e2d\uff0c\u4f5c\u8005\u5c06\u5b83\u5206\u4e3a\u4e86\u4e09\u79cd\u7c7b\u578b\uff1adiscrete alignment\uff08\u5bf9\u79bb\u6563\u6570\u636e\u8fdb\u884c\u5bf9\u9f50\uff09\uff0ccontinuous alighment\uff08\u5373\u5bf9\u8fde\u7eed\u7684\u6570\u636e\u8fdb\u884c\u5bf9\u9f50\uff0c\u8fd9\u66f4\u591a\u662f\u52a0\u6743\u5173\u7cfb\uff09\uff0ccontextualized representations\uff08\u5373\u901a\u8fc7\u6355\u6349\u5143\u7d20\u4e4b\u95f4\u7684\u8de8\u6a21\u6001\u4ea4\u4e92\u6765\u5f97\u5230\u66f4\u597d\u7684\u8bad\u7ec3\u6548\u679c\uff09<\/li>\n<\/ul>\n\n\n\n<figure class=\"wp-block-image size-full\"><div class='fancybox-wrapper lazyload-container-unload' data-fancybox='post-images' href='https:\/\/blogqian.com\/wp-content\/uploads\/2025\/07\/\u622a\u5c4f2025-04-20-02.09.14.png'><img class=\"lazyload lazyload-style-8\" src=\"data:image\/svg+xml;base64,PCEtLUFyZ29uTG9hZGluZy0tPgo8c3ZnIHdpZHRoPSIxIiBoZWlnaHQ9IjEiIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8yMDAwL3N2ZyIgc3Ryb2tlPSIjZmZmZmZmMDAiPjxnPjwvZz4KPC9zdmc+\"  loading=\"lazy\" decoding=\"async\" width=\"2206\" height=\"426\" data-original=\"https:\/\/blogqian.com\/wp-content\/uploads\/2025\/07\/\u622a\u5c4f2025-04-20-02.09.14.png\" src=\"data:image\/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAAJcEhZcwAADsQAAA7EAZUrDhsAAAANSURBVBhXYzh8+PB\/AAffA0nNPuCLAAAAAElFTkSuQmCC\" alt=\"\" class=\"wp-image-580\"  sizes=\"auto, (max-width: 2206px) 100vw, 2206px\" \/><\/div><\/figure>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Resoning\uff1a\u5728\u591a\u6a21\u6001\u8bad\u7ec3\u4e2d\uff0c\u5f80\u5f80\u9700\u8981\u901a\u8fc7\u591a\u4e2a\u6b65\u9aa4\uff0c\u6765\u878d\u5165\u4e0d\u540cmodality\u7684\u77e5\u8bc6\u548c\u6570\u636e\uff08\u751a\u81f3\u5728\u6709\u7684\u60c5\u51b5\u4e0b\u52a0\u5165\u4e00\u4e9bExternal Knowledge\u6765\u63d0\u5347model\u6027\u80fd\uff09\u3002<\/li>\n<\/ul>\n\n\n\n<figure class=\"wp-block-image size-full\"><div class='fancybox-wrapper lazyload-container-unload' data-fancybox='post-images' href='https:\/\/blogqian.com\/wp-content\/uploads\/2025\/07\/\u622a\u5c4f2025-04-20-02.14.52.png'><img class=\"lazyload lazyload-style-8\" src=\"data:image\/svg+xml;base64,PCEtLUFyZ29uTG9hZGluZy0tPgo8c3ZnIHdpZHRoPSIxIiBoZWlnaHQ9IjEiIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8yMDAwL3N2ZyIgc3Ryb2tlPSIjZmZmZmZmMDAiPjxnPjwvZz4KPC9zdmc+\"  loading=\"lazy\" decoding=\"async\" width=\"2206\" height=\"436\" data-original=\"https:\/\/blogqian.com\/wp-content\/uploads\/2025\/07\/\u622a\u5c4f2025-04-20-02.14.52.png\" src=\"data:image\/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAAJcEhZcwAADsQAAA7EAZUrDhsAAAANSURBVBhXYzh8+PB\/AAffA0nNPuCLAAAAAElFTkSuQmCC\" alt=\"\" class=\"wp-image-581\"  sizes=\"auto, (max-width: 2206px) 100vw, 2206px\" \/><\/div><\/figure>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Generation\uff1a\u5373\u8bad\u7ec3\u4e00\u79cd\u751f\u6210\u8fc7\u7a0b\uff0c\u4ee5\u6b64\u751f\u4ea7\u4e00\u4e9b\u65b0\u7684modality\uff0c\u7531\u6e90\u4fe1\u606f\u548c\u751f\u6210\u4fe1\u606f\u7684\u591a\u5be1\u53ef\u5206\u4e3aSummarization, Translation, Creation:<\/li>\n<\/ul>\n\n\n\n<figure class=\"wp-block-image size-full\"><div class='fancybox-wrapper lazyload-container-unload' data-fancybox='post-images' href='https:\/\/blogqian.com\/wp-content\/uploads\/2025\/07\/\u622a\u5c4f2025-04-20-02.25.01.png'><img class=\"lazyload lazyload-style-8\" src=\"data:image\/svg+xml;base64,PCEtLUFyZ29uTG9hZGluZy0tPgo8c3ZnIHdpZHRoPSIxIiBoZWlnaHQ9IjEiIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8yMDAwL3N2ZyIgc3Ryb2tlPSIjZmZmZmZmMDAiPjxnPjwvZz4KPC9zdmc+\"  loading=\"lazy\" decoding=\"async\" width=\"2206\" height=\"376\" data-original=\"https:\/\/blogqian.com\/wp-content\/uploads\/2025\/07\/\u622a\u5c4f2025-04-20-02.25.01.png\" src=\"data:image\/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAAJcEhZcwAADsQAAA7EAZUrDhsAAAANSURBVBhXYzh8+PB\/AAffA0nNPuCLAAAAAElFTkSuQmCC\" alt=\"\" class=\"wp-image-582\"  sizes=\"auto, (max-width: 2206px) 100vw, 2206px\" \/><\/div><\/figure>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Transfer\uff1a\u5373\u5728\u4e0d\u540c\u7684modalities\u4e4b\u95f4\u4f20\u9012knowledge\uff0c\u4e00\u822c\u6765\u8bf4\u662f\u4e3a\u4e86\u5e2e\u52a9\u4e00\u4e9b\u8bad\u7ec3\u6570\u636e\u8d44\u6e90\u6709\u9650\u6216\u8005noisier\u7684modality\u83b7\u5f97\u66f4\u597d\u7684\u8868\u73b0\u3002\uff08e.g. \u5728\u4e0a\u53e4\u65f6\u4ee3\u7684\u7814\u7a76\u4e2d\uff082018\u5e74\u4e4b\u524d\uff09\u4eba\u4eec\u4e00\u76f4\u5c1d\u8bd5\u5c06CV\u9886\u57df\u7684model\u4e2d\u7684Knowledge\u4f20\u9012\u5230\u8bed\u8a00\u6a21\u578b\u4e2d\uff0c\u4f46\u5728\u5982\u4eca\u8fd9\u4e2a\u65f6\u4ee3\uff0c\u7531\u4e8e\u5927\u8bed\u8a00\u6a21\u578b\u4e2d\u6709\u5de8\u91cf\u7684\u4fe1\u606f\uff0c\u6240\u4ee5\u6700\u8fd1\u7684\u7814\u7a76\u57fa\u672c\u4e0a\u7740\u773c\u4e8e\u5c06LLM\u7684\u4fe1\u606f\u4f20\u9012\u5230\u5176\u4ed6model\u4e2d<a href=\"#id1\">[1]<\/a>\uff09<\/li>\n<\/ul>\n\n\n\n<figure class=\"wp-block-image size-full\"><div class='fancybox-wrapper lazyload-container-unload' data-fancybox='post-images' href='https:\/\/blogqian.com\/wp-content\/uploads\/2025\/07\/\u622a\u5c4f2025-04-20-02.32.31.png'><img class=\"lazyload lazyload-style-8\" src=\"data:image\/svg+xml;base64,PCEtLUFyZ29uTG9hZGluZy0tPgo8c3ZnIHdpZHRoPSIxIiBoZWlnaHQ9IjEiIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8yMDAwL3N2ZyIgc3Ryb2tlPSIjZmZmZmZmMDAiPjxnPjwvZz4KPC9zdmc+\"  loading=\"lazy\" decoding=\"async\" width=\"2206\" height=\"406\" data-original=\"https:\/\/blogqian.com\/wp-content\/uploads\/2025\/07\/\u622a\u5c4f2025-04-20-02.32.31.png\" src=\"data:image\/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAAJcEhZcwAADsQAAA7EAZUrDhsAAAANSURBVBhXYzh8+PB\/AAffA0nNPuCLAAAAAElFTkSuQmCC\" alt=\"\" class=\"wp-image-583\"  sizes=\"auto, (max-width: 2206px) 100vw, 2206px\" \/><\/div><\/figure>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Quantification\uff1a\u5373\u91cf\u5316\uff0c\u901a\u8fc7\u91cf\u5316\uff0c\u6211\u4eec\u4e0d\u53bb\u8ffd\u6c42better accuracy\uff0c\u800c\u662f\u5728\u91cf\u5316\u540e\u7684\u8bad\u7ec3\u548c\u7814\u7a76\u4e2d\u80fd\u591f\u66f4\u597d\u7684\u7406\u89e3Multimodal\u7684\u672c\u8d28<\/li>\n<\/ul>\n\n\n\n<p>\u8fd9\u91cc\u6211\u5df2\u7ecf\u4ece\u5b8f\u89c2\u89d2\u5ea6\u9610\u8ff0\u4e86\u591a\u6a21\u6001\u8fd9\u4e2a\u9886\u57df\uff0c\u63a5\u4e0b\u6765\u6211\u4f1a\u8c08\u4e00\u8c08\u6211\u5bf9\u4e8e\u7ec6\u5206\u9886\u57df\u2014\u2014\u591a\u6a21\u6001\u7684\u5782\u57df\u5e94\u7528\u5fae\u8c03\u2014\u2014\u7684\u7406\u89e3\u3002<\/p>\n\n\n\n<p>\u5bf9\u4e8e\u4e00\u4e2a\u5df2\u7ecf\u8bad\u7ec3\u597d\u7684\u4e0a\u6e38\u6a21\u578b\uff0c\u5728\u8fdb\u884c\u5b9e\u9645\u5de5\u7a0b\u843d\u5730\u7684\u65f6\u5019\u4f1a\u8fdb\u884c\u5fae\u8c03\uff0c\u4f46\u662f\u666e\u904d\u4e0a\uff0c\u4e0a\u6e38\u6a21\u578b\u5fae\u8c03\u5b58\u5728\u4ee5\u4e0b\u95ee\u9898<a href=\"#id6\">[6]<\/a>\uff1a<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li><strong>Task-Expert Specialization<\/strong> \uff1a\u5f53\u76ee\u6807\u7684\u6570\u636e\u96c6\u548c\u9884\u5148\u8bad\u7ec3\u7684\u6570\u636e\u96c6\u5728\u5206\u5e03\u4e0a\u6709\u8f83\u5927\u5dee\u5f02\u7684\u65f6\u5019\uff0cMLLM\u6a21\u578b\u6027\u80fd\u5c06\u53d7\u5230\u9650\u5236\u3002<\/li>\n\n\n\n<li><strong>Open-World Stabilization<\/strong> \uff1a \u5373\u6a21\u578b\u5728\u4e0b\u6e38\u8c03\u4f18\u7684\u65f6\u5019\u53ef\u80fd\u4f1a\u9762\u4e34\u707e\u96be\u6027\u7684\u9057\u5fd8\uff0c\u4ece\u800c\u4e22\u5931\u9884\u8bad\u7ec3\u65f6\u83b7\u5f97\u7684\u901a\u7528\u77e5\u8bc6\u3002\u4e3a\u4e86\u89e3\u51b3\u8fd9\u4e9b\u95ee\u9898\uff0c\u8c03\u4f18\u8303\u5f0f\u53ef\u4ee5\u88ab\u603b\u7ed3\u6210\u4e09\u79cd<a href=\"#id6\">[6]<\/a>\uff1a<\/li>\n\n\n\n<li>Selective Tuning\uff1a\u5373\u5206\u8fa8\u51fa\u4e0e\u4e0b\u6e38\u4efb\u52a1\u76f8\u5173\u7684\u53c2\u6570\u662f\u54ea\u4e9b\uff0c\u4ec5\u4ec5\u8c03\u6574\u8fd9\u90e8\u5206\u53c2\u6570\uff0c\u4fdd\u6301\u5176\u4ed6\u53c2\u6570\u4e0d\u53d8\u3002\uff08e.g. SPIDER\u65b9\u6cd5\uff0c\u901a\u8fc7PID\u52a8\u6001\u9009\u62e9\u5173\u952e\u53c2\u6570\u8fdb\u884c\u66f4\u65b0<a href=\"#id7\">[7]<\/a>\uff09<\/li>\n\n\n\n<li>Additive Tuning\uff1a\u5373\u56fa\u5b9a\u5df2\u6709\u7684\u53c2\u6570\u548c\u5c42\u4e0d\u52a8\uff0c\u6dfb\u52a0\u4e00\u4e9b\u53ef\u8bad\u7ec3\u6a21\u5757\u4ec5\u4ec5\u53bb\u4f18\u5316\u548c\u8c03\u6574\u9644\u52a0\u7684\u90e8\u5206\u3002<\/li>\n\n\n\n<li>Reparameterization Tuning\uff1a\u5c06\u6743\u91cd\u77e9\u9635\u5206\u89e3\u4e3a\u4f4e\u79e9\u77e9\u9635\u7684\u4e58\u79ef\uff0c\u4ec5\u4ec5\u66f4\u65b0\u4f4e\u79e9\u5206\u91cf\uff0c\u6765\u9632\u6b62\u5168\u5c40\u9057\u5fd8\u548c\u6a21\u578b\u6027\u80fd\u4e0b\u964d\uff08e.g. LoraSculpt\u6846\u67b6<a href=\"#id8\">[8]<\/a>\uff09\u3002<\/li>\n<\/ol>\n\n\n\n<h1 class=\"wp-block-heading\">\u53c2\u8003\u6587\u732e<\/h1>\n\n\n\n<p><a>[1]<\/a>Liang, Paul Pu, Amir Zadeh, and Louis-Philippe Morency. &#8220;Foundations and trends in multimodal machine learning: Principles, challenges, and open questions.&#8221; <em>arXiv preprint arXiv:2209.03430<\/em> (2022).<\/p>\n\n\n\n<p><a>[2]<\/a>Baltru\u0161aitis, T., Ahuja, C., &amp; Morency, L. P. (2018). Multimodal Machine Learning: A Survey and Taxonomy. IEEE Transactions on Pattern Analysis and Machine Intelligence.<\/p>\n\n\n\n<p><a>[3]<\/a>Zi-Yi Dou, Yichong Xu, Zhe Gan, Jianfeng Wang, Shuohang Wang, Lijuan Wang, Chenguang Zhu, Pengchuan Zhang, Lu Yuan, Nanyun Peng, Zicheng Liu, and Michael Zeng. An empirical study of training end-to-end vision-and-language transformers, 2021<\/p>\n\n\n\n<p><a>[4]<\/a> Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision, 2021.<\/p>\n\n\n\n<p><a>[5]<\/a>Junnan Li, Ramprasaath R. Selvaraju, Akhilesh Deepak Gotmare, Shafiq Joty, Caiming Xiong, and Steven Hoi. Align before fuse: Vision and language representation learning with momentum distillation, 2021<\/p>\n\n\n\n<p><a>[6]<\/a>Huang, Wenke, et al. &#8220;Keeping yourself is important in downstream tuning multimodal large language model.&#8221; <em>arXiv preprint arXiv:2503.04543<\/em> (2025).<\/p>\n\n\n\n<p><a>[7]<\/a>Huang, Wenke, et al. &#8220;Learn from Downstream and Be Yourself in Multimodal Large Language Model Fine-Tuning.&#8221; <em>arXiv preprint arXiv:2411.10928<\/em> (2024).<\/p>\n\n\n\n<p><a>[8]<\/a>Liang, Jian, et al. &#8220;Lorasculpt: Sculpting lora for harmonizing general and specialized knowledge in multimodal large language models.&#8221; <em>arXiv preprint arXiv:2503.16843<\/em> (2025).<\/p>\n","protected":false},"excerpt":{"rendered":"<p>\u5bf9\u4e8e\u76ee\u524d\u672c\u4eba\u4e3b\u8981\u7814\u7a76\u65b9\u5411\u7684\u7efc\u8ff0\u6027\u603b\u7ed3<\/p>\n","protected":false},"author":1,"featured_media":577,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[12],"tags":[34],"class_list":["post-576","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-research","tag-multimodality"],"_links":{"self":[{"href":"https:\/\/blogqian.com\/index.php\/wp-json\/wp\/v2\/posts\/576","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blogqian.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogqian.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogqian.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/blogqian.com\/index.php\/wp-json\/wp\/v2\/comments?post=576"}],"version-history":[{"count":1,"href":"https:\/\/blogqian.com\/index.php\/wp-json\/wp\/v2\/posts\/576\/revisions"}],"predecessor-version":[{"id":584,"href":"https:\/\/blogqian.com\/index.php\/wp-json\/wp\/v2\/posts\/576\/revisions\/584"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blogqian.com\/index.php\/wp-json\/wp\/v2\/media\/577"}],"wp:attachment":[{"href":"https:\/\/blogqian.com\/index.php\/wp-json\/wp\/v2\/media?parent=576"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogqian.com\/index.php\/wp-json\/wp\/v2\/categories?post=576"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogqian.com\/index.php\/wp-json\/wp\/v2\/tags?post=576"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}