| GitHub Launches AI Models Sandbox |
| Written by Kay Ewbank |
| Monday, 05 August 2024 |
|
GitHub has launched a playground for developers to try AI models. GitHub Models can be used to test and compare different models. GitHub says the tool has been developed because "a vast segment of developers still lack easy access to open and closed models".
The models available in the sandbox include Llama 3.1, GPT-4o and GPT-4o mini, Phi 3 and Mistral Large 2. The tool lets developers test different prompts and model parameters for free in GitHub. There's then a "glide path" to bring the models to developer environments in Codespaces and VS Code, with a final route through to Azure AI for production where developers can switch to the free GitHub token to a token from a paid Azure account. The Codespaces option comes with sample code for a variety of languages and frameworks, with the aim of letting developers try out various scenarios. This can then be used to set up a prototype or proof-of-concept within developers' own applications. The models are available in GitHub Marketplace, where developers can adjust model parameters and submit prompts to see how the model responds, and can see the code that corresponds to the parameters that have been selected. The playground is rate limited. Alternatively, developers can run prompt evals in GitHub Actions with a series of JSON files by piping in the GitHub Models command within the GitHub CLI. The sandbox also offers free API usage so that developers can experiment with AI models in their own applications, choosing the language they want to use, and which SDK to use. All the models can be used with the Azure AI Inference SDK, and some models support additional SDKs. The free API usage is also rate limited. The playground and free API usage are rate limited by requests per minute, requests per day, tokens per request, and concurrent requests. If you get rate limited, you will need to wait for the rate limit that you hit to reset before you can make more requests. Low, high, and embedding models have different rate limits. GitHub Models is available in limited beta now.
More InformationRelated ArticlesGitHub Adds Required Workflows For Actions GitHub Copilot Provides Productivity Boost GitHub Desktop 2.0 Introduces Stashing and Rebasing To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin. 
Comments
or email your comment to: comments@i-programmer.info |
