
StarCoder - ã³ãŒãã®ããã®LLM
以äžã®èšäºãé¢çœãã£ãã®ã§ãç°¡åã«ãŸãšããŸããã
1. BigCode
ãBigCodeãã¯ããHuggingFaceããšãServiceNowããå ±åã§äž»å°ãããªãŒãã³ãªã³ã©ãã¬ãŒã·ã§ã³ã§ããã³ãŒãã®ããã®LLMã®è²¬ä»»ããéçºã«åãçµãã§ããŸãã
2. StarCoder
ãStarCoderããšãStarCoderBaseãã¯ã80以äžã®ããã°ã©ãã³ã°èšèªãGitã³ããããGitHub issueãJupyter notebookãªã©ãGitHubããèš±å¯ãããããŒã¿ã§åŠç¿ããã³ãŒãã®ããã®LLM (Code LLM) ã§ãããStarCoderBaseãã¯15Bãã©ã¡ãŒã¿ã¢ãã«ã1å ããŒã¯ã³ã§åŠç¿ããStarCoderãã¯ãStarCoderBaseãã35BããŒã¯ã³ã§ãã¡ã€ã³ãã¥ãŒãã³ã°ããã¢ãã«ã«ãªããŸãã
ãStarCoderãã¯ä»ã®ãªãŒãã³ãªLLMãããå€ãã®å ¥åãåŠçã§ã (8,000ããŒã¯ã³ä»¥äž)ãå¹ åºãã¢ããªã±ãŒã·ã§ã³ãå¯èœã«ããŸããããã«ãã£ãŠãã³ãŒãããªãŒãã³ã³ããªãŒãããããæç€ºãä»ããŠã³ãŒãã倿Žããããèªç¶èšèªã§ã³ãŒãã説æãããããããã«äœ¿çšã§ããŸãã
3. è©äŸ¡
ãStarCoderããšããã€ãã®é¡äŒŒã¢ãã«ããæ§ã ãªãã³ãããŒã¯ã§è©äŸ¡ããŸããããStarCoderããStarCoderBaseãã®äž¡æ¹ããã¢ãã«ãµã€ãºãå°ããã«ããããããããPaLMããLaMDAããLLaMAããªã©ã®å€§ããªã¢ãã«ãããåªããŠããããšãããããŸããã
ã»HumanEval
ã»MBPP (Multi-lingual Back-Translation Precision)

4. æè¡ã¢ã·ã¹ã¿ã³ã
ãStarCoderãããæè¡ã¢ã·ã¹ã¿ã³ãããšããŠäœ¿çšã§ãããã©ããããã¹ãããŸãããããã¯å€ãã®ããã¥ã¡ã³ããšGitHub issueã§åŠç¿ããŸãããAnthropicã®HHHããã³ããã«è§ŠçºãããTech Assistantããã³ãããæ§ç¯ããŸããã
ã»HHHããã³ãã (翻蚳)
以äžã¯ãããŸããŸãªäººãšAIã¢ã·ã¹ã¿ã³ããšã®äžé£ã®å¯Ÿè©±ã§ãã AIã¯ã芪åã§ãäžå¯§ã§ãæ£çŽã§ãæŽç·ŽãããŠããŠãææ
çã§ãè¬èã§ãã§ãç¥èãè±å¯ã§ããããã«åªããŠããŸãã ã¢ã·ã¹ã¿ã³ãã¯ãã»ãšãã©äœã§ãåãã§ãæäŒãããŸãããäœãå¿
èŠããæ£ç¢ºã«çè§£ããããã«æåãå°œãããŸãã ãŸãã誀ã£ãæ
å ±ã誀解ãæããããªæ
å ±ãäžããªãããã«ããæ£ããçããå®å
šã«ããããªãå Žåã¯ããã®æšã説æããŸãã ãšã¯ããããã®ã¢ã·ã¹ã¿ã³ãã¯å®çšçã§ãæ¬åœã«ãã¹ããå°œãããŠãããŸãããèŠæå¿ãéªéããããããªããšããªãã®ã§ãå®å¿ããŠäœ¿ããŸãã
-----
Human: æ°ã«ãªã£ãã®ã§ãããå®ã¯ã¹ãã²ããã£ãäœããšãã«å¡©ãå
¥ããããšã¯éèŠãªã®ã§ããããïŒ
Assistant: ã¹ãã²ããã£ãè¹ã§ããæ¹¯ã«å¡©ãå
¥ããã®ã¯ éèŠãªãã§ããããïŒ
Human: ã¯ã
Assistant: éåžžããã¹ã¿ã®å³ãåŒãåºãããã«æšå¥šãããŠãããã®ã§ãã ç§åŠçã«èšããšãå¡©ã¯èã«ããååŠå容åšãå³èŠã§æããååãæç¥ããèœåãé«ãããšããããšãªãã§ãã
Human: ã§ãããœãŒã¹ãšãã䜿ãããé¢ä¿ãªãããªïŒ
Assistant: åä¿¡åçã®ããã§ããã æ£çŽãªãšãããã¯ã£ããããããšã¯ããããŸããããèå³ãããããªã®ã§ããèªåã§å³èŠãã¹ããããŠã¿ãŠã¯ãããã§ãããããå°ããªéã«ãã¹ã¿ã2ã€äœããäžæ¹ã«ã¯å¡©ããã£ã·ãå
¥ããããäžæ¹ã«ã¯å¡©ãå
¥ããªãã ãããŠãã©ã¡ãã®éººã奜ãããå³èŠãããŠã¿ãŠãã ããã ãŸãããã£ãããããã«ãäž¡æ¹ã«ãœãŒã¹ããããŠããã¬ãŒã³ãªãã¹ã¿ã§æããå³ã®éããæããããªããªããã©ããã確ãããŠã¿ãŠãã ããã æ¥œããå®éšã«ãªãã¯ãã§ãïŒ
-----ã»Tech Assistantããã³ãã (翻蚳)
以äžã¯ãããŸããŸãªäººã
ãš AI æè¡ã¢ã·ã¹ã¿ã³ããšã®éã®äžé£ã®å¯Ÿè©±ã§ãã ã¢ã·ã¹ã¿ã³ãã¯ã圹ã«ç«ã¡ãç€Œåæ£ãããæ£çŽã§ãæŽç·Žãããææ
ãæèããè¬èã§ãããªããç¥èãè±å¯ã§ããããåªããŸãã ã¢ã·ã¹ã¿ã³ãã¯åãã§ã³ãŒãã«é¢ãã質åã«çããå¿
èŠãªãã®ãæ£ç¢ºã«çè§£ããããã«æåãå°œãããŸãã ãŸããèåœãŸãã¯èª€è§£ãæããããªæ
å ±ãæäŸããªãããã«åªããæ£ããçããå®å
šã«ããããªãå Žåã¯æ³šæãä¿ããŸãã ãšã¯ãããã¢ã·ã¹ã¿ã³ãã¯å®çšçã§ãããæ¬åœã«æåãå°œãããŠãããæçšã§ããããšã«æ³šæãæããããªãããã«ããŠããŸã.
-----
Human: 2 ã€ã®ãªã¹ããåãåããåå
¥åãªã¹ããã亀äºã«èŠçŽ ãå«ããªã¹ããè¿ã颿°ãäœæããŸãã
Assistant: ããããŸããã ãããè¡ã颿°ã次ã«ç€ºããŸãã
```
def alternating(list1, list2):
results = []
for i in range(len(list1)):
results.append(list1[i])
results.append(list2[i])
return results
```
Human: ãã®é¢æ°ã®ãã¹ã ã±ãŒã¹ãæžããŠããããŸãã?
Assistant: ã¯ããããã€ãã®ãã¹ãããããŸãã
```
assert alternating([10, 20, 30], [1, 2, 3]) == [10, 1, 20, 2, 30, 3]
assert alternating([True, False], [4, 5]) == [True, 4, False, 5]
assert alternating([], []) == []
```
Human: ãªã¹ãã®é·ããäžåäžãªå Žåã«ãã¹ãŠã®å
¥åèŠçŽ ãè¿ãããã«é¢æ°ã倿ŽããŸãã é·ããªã¹ãã®èŠçŽ ã¯æåŸã«ããå¿
èŠããããŸãã
Assistant: ããã倿Žããã颿°ã§ãã
```"
def alternating(list1, list2):
results = []
for i in range(min(len(list1), len(list2))):
results.append(list1[i])
results.append(list2[i])
if len(list1) > len(list2):
results.extend(list1[i+1:])
else:
results.extend(list2[i+1:])
return results
```
-----ããã³ããã ãã§ãã¢ãã«ã¯æè¡ã¢ã·ã¹ã¿ã³ããšããŠæ©èœããããã°ã©ãã³ã°é¢é£ã®èŠæ±ã«çããããšãããããŸããã

5. åŠç¿ããŒã¿
ãStarCoderãã¯ããThe Stack 1.2ãã®ãµãã»ããã§åŠç¿ãããŠããŸããããŒã¿ã»ããã¯ãèš±å¯ãããã³ãŒãã®ã¿ã§æ§æãããã³ãŒãè²¢ç®è ãããŒã¿ã»ããããããŒã¿ãåé€ã§ããããã«ãªããã¢ãŠãããããã»ã¹ãå«ãŸããŠããŸããããã«ãåŠç¿ããŒã¿ããå人ãç¹å®ã§ããæ å ± (ååããã¹ã¯ãŒããã¡ãŒã«ã¢ãã¬ã¹ãªã©)ãåé€ããŠããŸãã
6. 远å ãªãªãŒã¹
ã¢ãã«ãšå ±ã«ããªãœãŒã¹ãšãã¢ã®ãªã¹ãããªãªãŒã¹ããŠããŸãã
ã»OpenRAILã©ã€ã»ã³ã¹ã®äžéãã§ãã¯ãã€ã³ããå«ãã¢ãã«ãŠã§ã€ã
ã»Apache 2.0ã©ã€ã»ã³ã¹ã«ããããŒã¿ååŠçãšåŠç¿ã®ããã®å šã³ãŒã
ã»ã³ãŒãã¢ãã«ã®å æ¬çãªè©äŸ¡ããŒãã¹
ã»PIIé€å»ã®åŠç¿ãšè©äŸ¡ã®ããã®æ°ããPIIããŒã¿ã»ãã
ã»åŠç¿ã«äœ¿çšãããååŠçãããããŒã¿ã»ãã
ã»ããŒã¿ã»ããã§çæãããã³ãŒããèŠã€ããããã®ã³ãŒãã¢ããªãã¥ãŒã·ã§ã³ããŒã«
7. ãªã³ã¯
7.1 ã¢ãã«
ã»è«æ
ã»GitHub
ã»StarCoder
ã»StarCoderBase
ã»StarEncoder
ã»StarPii
7.2 ããŒã«ãšãã¢
ã»StarCoder Chat
ã»VSCodeæ¡åŒµæ©èœ
ã»StarCoder PlaygroundÂ
ã»StarCoderãšãã£ã¿
7.3 ããŒã¿ãšã¬ããã³ã¹
ã»StarCoderData
ã»æè¡ã¢ã·ã¹ã¿ã³ãããã³ãã
ã»ã¬ããã³ã¹ã«ãŒã
ã»StarCoderã©ã€ã»ã³ã¹å¥çŽ
ã»StarCoderæ€çŽ¢
ã»StarCoderã¡ã³ããŒã·ãããã¹ã