Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

.Net: Bug: Cannot execute KernelFunctions when using - phi-3 cpu-int4-rtn-block-32 #9798

Closed
Vizeo opened this issue Nov 23, 2024 · 1 comment
Labels
bug Something isn't working .NET Issue or Pull requests regarding .NET code triage

Comments

@Vizeo
Copy link

Vizeo commented Nov 23, 2024

Cannot execute KernelFunctions when using - phi-3 cpu-int4-rtn-block-32.

Running the following code should return "Ironman". But the function never gets called.

To Reproduce
Run Code
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging;
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.ChatCompletion;
using Microsoft.SemanticKernel.Connectors.OpenAI;
using System.ComponentModel;

namespace AI_test
{
internal class Program
{
public class SuperHeroPlugin
{
[KernelFunction, Description("The best super hero")]
public string TheBestSuperHero() => "Ironman";
}

	static async Task Main(string[] args)
	{
		// Your PHI-3 model location 
		var modelPath = @"D:\AI Models\Phi-3\cpu_and_mobile\cpu-int4-rtn-block-32";

		// Load the model and services
		var builder = Kernel.CreateBuilder();
		builder.AddOnnxRuntimeGenAIChatCompletion("phi-3", modelPath);
		builder.Services.AddLogging(c => c.AddConsole().SetMinimumLevel(LogLevel.Trace));
		builder.Plugins.AddFromType<SuperHeroPlugin>();
		var kernel = builder.Build();

		// Create services such as chatCompletionService and embeddingGeneration
		var chatCompletionService = kernel.GetRequiredService<IChatCompletionService>();
		ChatHistory chat = new();
		OpenAIPromptExecutionSettings settings = new() { ToolCallBehavior = ToolCallBehavior.AutoInvokeKernelFunctions };

		// Start the conversation
		Console.Write("Who is the best superhero?");
		var question = "Who is the best superhero?";

		ChatMessageContent chatResult = await chatCompletionService.GetChatMessageContentAsync(question, settings, kernel);

		Console.Write($"\n>>> Result: {chatResult}\n\n> ");
	}
}

Expected behavior
returns "Ironman"

Platform

  • OS: Windows
  • IDE: Visual Studio 2022
  • Language: C#
@Vizeo Vizeo added the bug Something isn't working label Nov 23, 2024
@markwallace-microsoft markwallace-microsoft added .NET Issue or Pull requests regarding .NET code triage labels Nov 23, 2024
@github-actions github-actions bot changed the title Bug: Cannot execute KernelFunctions when using - phi-3 cpu-int4-rtn-block-32 .Net: Bug: Cannot execute KernelFunctions when using - phi-3 cpu-int4-rtn-block-32 Nov 23, 2024
@markwallace-microsoft
Copy link
Member

@Vizeo This is currently not supported by ONNX GenAI, these are the tracking issues

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working .NET Issue or Pull requests regarding .NET code triage
Projects
None yet
Development

No branches or pull requests

2 participants