Qwen Coder 2.5 32b on M4 Max
✎ Published:
Initial notes on how Qwen Coder 2.5 performs on a Macbook Pro M4 Max
I received my new Macbook Pro M4 Max today and at the same time Qwen released their new Coder 2.5 model.
I quickly installed LMStudio, downloaded the GGUF models and ran some tests.
I used two different GGUF models for these tests:
- Qwen Coder 2.5 32b q8
- Qwen Coder 2.5 32b q2_k
I chose these two to observer the difference in performance between lightly and heavily quantized models.
Environment
Those were loaded with the following settings:
- Context Length: 4096k
- Flash Attention: Disabled
- GPU Offload: Full
- CPU Thread Pool Size: 12
- Evaluation Batch Size 512
- Keep Model in Memory: True
- Try mmap: True
First Prompt
Results on this prompt (48 tokens):
I have this react fn in typescript
export function LoginForm()
how can I change it so that it can receive these two functions as parameters:
export async function signup(formData: FormData) {}
export async function login(formData: FormData) {}(Note that the prompt is deliberately not well written)
q8
- Response: 12.96 tok/sec, 564 tokens, 1.02s to first token
import React from 'react';
// Define the interface for the LoginForm props
interface LoginFormProps {
signup: (formData: FormData) => Promise<void>;
login: (formData: FormData) => Promise<void>;
}
export function LoginForm({ signup, login }: LoginFormProps) {
// Example usage of the signup and login functions within the component
const handleSubmit = async (event: React.FormEvent<HTMLFormElement>) => {
event.preventDefault();
const formData = new FormData(event.currentTarget);
// Determine if it's a signup or login attempt
// This is just an example, you might want to use another mechanism to determine this.
const isSignupAttempt = true; // Change this based on your logic
try {
if (isSignupAttempt) {
await signup(formData);
} else {
await login(formData);
}
} catch (error) {
console.error('Error during form submission:', error);
}
};
return (
<form onSubmit={handleSubmit}>
{/* Your form fields go here */}
<button type="submit">Submit</button>
</form>
);
}
// Example usage of LoginForm component
export async function signup(formData: FormData) {
// Signup logic here
console.log('Signing up with:', formData);
}
export async function login(formData: FormData) {
// Login logic here
console.log('Logging in with:', formData);
}
<LoginForm signup={signup} login={login} />q2_k
- Response: 17.95 tok/sec, 494 tokens, 0.76s to first token
import React from 'react';
// Define the type for the props that LoginForm will receive
interface LoginFormProps {
signup: (formData: FormData) => Promise<void>;
login: (formData: FormData) => Promise<void>;
}
export function LoginForm({ signup, login }: LoginFormProps) {
// Your form rendering logic here
const handleSubmit = async (event: React.FormEvent<HTMLFormElement>) => {
event.preventDefault();
const formData = new FormData(event.currentTarget);
// Example of calling the passed functions
if (someCondition) { // Replace with your actual condition to decide between signup and login
await signup(formData);
} else {
await login(formData);
}
};
return (
<form onSubmit={handleSubmit}>
{/* Your form fields here */}
<button type="submit">Submit</button>
</form>
);
}
// Example usage of LoginForm
// const App = () => {
// const handleSignup = async (formData: FormData) => {
// // signup logic
// };
// const handleLogin = async (formData: FormData) => {
// // login logic
// };
// return <LoginForm signup={handleSignup} login={handleLogin} />;
// };Quality
If you open the responses, you’ll see that the quality at q2_k really degrades. At the same time, the time difference of 12 to 17 seconds is not that long.
Second Prompt
Please find bugs in the following typescript code and explain in detail what the bug is and provide a git patch with a fix:
import React from 'react';
// Define the interface for the LoginForm props
interface LoginFormProps {
signup: (formData: FormData) => Promise<void>;
login: (formData: FormData) => Promise<void>;
}
export function LoginForm({ signup, login }: LoginFormProps) {
// Example usage of the signup and login functions within the component
const handleSubmit = async (event: React.FormEvent<HTMLFormElement>) => {
event.preventDefault();
const formData = new FormData(event.currentTarget);
// Determine if it's a signup or login attempt
// This is just an example, you might want to use another mechanism to determine this.
const isSignupAttempt = true; // Change this based on your logic
try {
if (isSignupAttempt) {
await signup(formData);
} else {
await signup(formData);
}
} catch (error) {
console.error('Error during form submission:', error);
}
};
return (
<form onSubmit={handleSubmit}>
{/* Your form fields go here */}
<button type="submitt">Submit</button>
</form>
);
}
// Example usage of LoginForm component
export async function signup(formData: FormData) {
// Signup logic here
console.log('Signing up with:', formData);
}
export async function login(formData: FormData) {
// Login logic here
console.log('Logging in with:', formData);
}
Results
q8
- Response: 12.72 tok/sec, 536 tokens, 2.20s to first token
### Bugs Identified:
1. **Incorrect Function Call**:
- In the `handleSubmit` function, when `isSignupAttempt` is `false`, it incorrectly calls `signup(formData)` instead of `login(formData)`.
2. **Typo in Button Type Attribute**:
- The `<button>` element has a typo in its `type` attribute: `submitt` should be `submit`.
### Detailed Explanation:
1. **Incorrect Function Call**:
- This bug will cause the application to always attempt to sign up, even if the intent is to log in. When `isSignupAttempt` is `false`, the code should call the `login` function instead of `signup`.
2. **Typo in Button Type Attribute**:
- The typo in the button type means that clicking the button will not trigger the form's `onSubmit` event handler as intended. The correct value for a submit button is `submit`.
### Git Patch with Fixes:
Here is the patch file (`0001-Fix-signup-login-call-and-typo-in-button-type.patch`) to address these issues.
diff --git a/src/LoginForm.tsx b/src/LoginForm.tsx
index 1234567..89abcde 100644
--- a/src/LoginForm.tsx
+++ b/src/LoginForm.tsx
@@ -20,7 +20,7 @@ export function LoginForm({ signup, login }: LoginFormProps) {
if (isSignupAttempt) {
await signup(formData);
} else {
- await signup(formData);
+ await login(formData);
}
} catch (error) {
console.error('Error during form submission:', error);
@@ -34,7 +34,7 @@ export function LoginForm({ signup, login }: LoginFormProps) {
</form>
);
}
-
+
// Example usage of LoginForm component
export async function signup(formData: FormData) {
// Signup logic here
### Applying the Patch:
1. Save the above content into a file named `0001-Fix-signup-login-call-and-typo-in-button-type.patch`.
2. Navigate to your project directory in the terminal.
3. Apply the patch using the `git apply` command:
git apply 0001-Fix-signup-login-call-and-typo-in-button-type.patch
After applying the patch, the code should work correctly by calling the appropriate function based on the value of `isSignupAttempt`, and the button will properly submit the form.q2_k
- Response: 17.95 tok/sec, 453 tokens, 2.08s to first token
The provided TypeScript code contains a few issues that need to be addressed. Let's go through each bug and provide the necessary fixes.
### Bugs Identified:
1. **Incorrect Function Call**: The `handleSubmit` function incorrectly calls the `signup` function twice instead of calling `login` for login attempts.
2. **Typo in Button Type Attribute**: The button type is misspelled as "submitt" instead of "submit".
Let's address these issues with a git patch.
### Git Patch
diff --git a/src/LoginForm.tsx b/src/LoginForm.tsx
index 1234567..89abcde 100644
--- a/src/LoginForm.tsx
+++ b/src/LoginForm.tsx
@@ -18,7 +18,7 @@ export function LoginForm({ signup, login }: LoginFormProps) {
try {
if (isSignupAttempt) {
await signup(formData);
- } else {
- await signup(formData);
+ } else {
+ await login(formData);
}
} catch (error) {
console.error('Error during form submission:', error);
@@ -30,7 +30,7 @@ export function LoginForm({ signup, login }: LoginFormProps) {
{/* Your form fields go here */}
- <button type="submitt">Submit</button>
+ <button type="submit">Submit</button>
</form>
);
}
### Explanation of Fixes:
1. **Correct Function Call**: The `handleSubmit` function now correctly calls the `login` function when it is not a signup attempt.
- await signup(formData);
+ await login(formData);
2. **Button Type Correction**: The button type attribute was corrected from "submitt" to "submit".
- <button type="submitt">Submit</button>
+ <button type="submit">Submit</button>
By applying this patch, the `LoginForm` component should work as intended, calling either the `signup` or `login` function based on the form submission context and correctly handling button submissions.Parameters
I also wanted to try the same prompt with different model settings. So here the prompt is as follows:
Can you explain what this code does and then convert it to swiftui?
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>3D Rotating Cube</title>
<style>
body {
display: flex;
justify-content: center;
align-items: center;
height: 100vh;
margin: 0;
background-color: #f0f0f0;
}
canvas {
background-color: #ffffff;
}
</style>
</head>
<body>
<canvas id="cubeCanvas" width="500" height="500"></canvas>
<script>
const canvas = document.getElementById("cubeCanvas");
const ctx = canvas.getContext("2d");
const width = canvas.width;
const height = canvas.height;
const halfWidth = width / 2;
const halfHeight = height / 2;
let angleX = 0;
let angleY = 0;
function drawCube() {
ctx.clearRect(0, 0, width, height);
const vertices = [
[-1, -1, -1], [1, -1, -1], [1, 1, -1], [-1, 1, -1],
[-1, -1, 1], [1, -1, 1], [1, 1, 1], [-1, 1, 1]
];
const edges = [
[0, 1], [1, 2], [2, 3], [3, 0],
[4, 5], [5, 6], [6, 7], [7, 4],
[0, 4], [1, 5], [2, 6], [3, 7]
];
const faces = [
[0, 1, 2, 3], [4, 5, 6, 7], [0, 1, 5, 4],
[2, 3, 7, 6], [1, 2, 6, 5], [0, 3, 7, 4]
];
const projectedVertices = vertices.map(v => {
let [x, y, z] = v;
// Rotate around X axis
let tempY = y * Math.cos(angleX) - z * Math.sin(angleX);
let tempZ = y * Math.sin(angleX) + z * Math.cos(angleX);
y = tempY;
z = tempZ;
// Rotate around Y axis
let tempX = x * Math.cos(angleY) - z * Math.sin(angleY);
z = x * Math.sin(angleY) + z * Math.cos(angleY);
x = tempX;
// Project the 3D coordinates to 2D
const scale = 200 / (z + 4);
const projX = x * scale + halfWidth;
const projY = y * scale + halfHeight;
return [projX, projY];
});
// Draw faces
ctx.fillStyle = "rgba(173, 216, 230, 0.5)"; // light blue
faces.forEach(face => {
ctx.beginPath();
ctx.moveTo(...projectedVertices[face[0]]);
for (let i = 1; i < face.length; i++) {
ctx.lineTo(...projectedVertices[face[i]]);
}
ctx.closePath();
ctx.fill();
});
// Draw edges
ctx.strokeStyle = "darkblue";
ctx.lineWidth = 2;
edges.forEach(edge => {
const [start, end] = edge;
ctx.beginPath();
ctx.moveTo(...projectedVertices[start]);
ctx.lineTo(...projectedVertices[end]);
ctx.stroke();
});
}
function animate() {
angleX += 0.01;
angleY += 0.01;
drawCube();
requestAnimationFrame(animate);
}
animate();
</script>
</body>
</html>Results
q2_k
- ~8k context, Enable Flash Attention
- 16.92 tok/sec, 859 tokens, 5.45s to first token
- Result didn’t compile in Xcode, has 4 errors which are probably easy to fix
import SwiftUI
import SceneKit
struct ContentView: View {
@State private var angleX: Float = 0.0
@State private var angleY: Float = 0.0
var body: some View {
VStack {
SCNVeviewContainer(angleX: $angleX, angleY: $angleY)
.frame(width: 500, height: 500)
Spacer()
}
.onAppear {
let _ = Timer.scheduling(timerTick, interval: 0.016) // Approximately 60 FPS
}
}
func timerTick(_ timer: Timer) {
angleX += 0.01
angleY += 0.01
}
}
struct SCNVeviewContainer: UIViewRepresentable {
@Binding var angleX: Float
@Binding var angleY: Float
let sceneView = SCNView()
func makeUIView(context: Context) -> SCNView {
// Set the background color of the scene view
sceneView.backgroundColor = UIColor.white
// Create a new SCNScene
let scene = SCNScene()
// Create a cube geometry
let cubeGeometry = SCNBox(width: 1.0, height: 1.0, length: 1.0, chamferRadius: 0)
cubeGeometry.firstMaterial?.diffuse.contents = UIColor(red: 0.68, green: 0.85, blue: 0.92, alpha: 0.5) // Light blue with transparency
// Create a node and add the geometry to it
let node = SCNNode(geometry: cubeGeometry)
// Set the initial position of the node
node.position = SCNVector3(0, 0, -4)
// Add the node to the scene
scene.rootNode.addChildNode(node)
// Set the scene to the view
sceneView.scene = scene
return sceneView
}
func updateUIView(_ uiView: SCNView, context: Context) {
let rotationX = SCNRotation(x: 1, y: 0, z: 0, angle: angleX)
let rotationY = SCNRotation(x: 0, y: 1, z: 0, angle: angleY)
// Apply the rotations to the node
uiView.scene!.rootNode.enumerateChildNodes { (node, stop) in
node.rotation = rotationX * rotationY
}
}
}
@main
struct ContentViewApp: App {
var body: some Scene {
WindowGroup {
ContentView()
.frame(width: 500, height: 500)
.background(Color(.lightGray))
}
}
}q2_k
- full context, Disable Flash Attention
- 15.22 tok/sec, 990 tokens, 6.95s to first token
- Result didn’t compile in Xcode, has 1 error (wrong pattern matching) which should be easy to fix
import SwiftUI
struct ContentView: View {
@State private var angleX: Double = 0
@State private var angleY: Double = 0
let vertices: [[Double]] = [
[-1, -1, -1], [1, -1, -1], [1, 1, -1], [-1, 1, -1],
[-1, -1, 1], [1, -1, 1], [1, 1, 1], [-1, 1, 1]
]
let edges: [[Int]] = [
[0, 1], [1, 2], [2, 3], [3, 0],
[4, 5], [5, 6], [6, 7], [7, 4],
[0, 4], [1, 5], [2, 6], [3, 7]
]
let faces: [[Int]] = [
[0, 1, 2, 3], [4, 5, 6, 7], [0, 1, 5, 4],
[2, 3, 7, 6], [1, 2, 6, 5], [0, 3, 7, 4]
]
var body: some View {
GeometryReader { geometry in
Path { path in
let width = geometry.size.width
let height = geometry.size.height
let halfWidth = width / 2
let halfHeight = height / 2
let projectedVertices = vertices.map { v in
let [x, y, z] = v
// Rotate around X axis
let tempY = y * cos(angleX) - z * sin(angleX)
let tempZ = y * sin(angleX) + z * cos(angleX)
// Rotate around Y axis
let tempX = x * cos(angleY) - tempZ * sin(angleY)
let finalZ = x * sin(angleY) + tempZ * cos(angleY)
// Project the 3D coordinates to 2D
let scale = 200 / (finalZ + 4)
let projX = tempX * scale + halfWidth
let projY = tempY * scale + halfHeight
return CGPoint(x: projX, y: projY)
}
// Draw faces
path.setFillColor(Color(red: 173/255, green: 216/255, blue: 230/255, opacity: 0.5))
for face in faces {
let points = face.map { projectedVertices[$0] }
path.addLines(points)
path.closeSubpath()
}
// Draw edges
path.setStrokeColor(Color.blue)
path.setLineWidth(2)
for edge in edges {
path.move(to: projectedVertices[edge[0]])
path.addLine(to: projectedVertices[edge[1]])
}
}
}
.onAppear {
Timer.scheduledTimer(withTimeInterval: 0.05, repeats: true) { _ in
self.angleX += 0.02
self.angleY += 0.02
}
}
.background(Color.gray)
}
}
struct ContentView_Previews: PreviewProvider {
static var previews: some View {
ContentView()
}
}q8
- No flash attention, 4k context
- 9.77 tok/sec, 1813 tokens, 5.58 to first token
- The result didn’t compile in Xcode, but I asked the model for the errors and it helped me fix them in another two iterations.
import SwiftUI
struct RotatingCube: View {
@State private var angleX = 0.0
@State private var angleY = 0.0
let vertices: [[Double]] = [
[-1, -1, -1], [1, -1, -1], [1, 1, -1], [-1, 1, -1],
[-1, -1, 1], [1, -1, 1], [1, 1, 1], [-1, 1, 1]
]
let edges: [[Int]] = [
[0, 1], [1, 2], [2, 3], [3, 0],
[4, 5], [5, 6], [6, 7], [7, 4],
[0, 4], [1, 5], [2, 6], [3, 7]
]
let faces: [[Int]] = [
[0, 1, 2, 3], [4, 5, 6, 7], [0, 1, 5, 4],
[2, 3, 7, 6], [1, 2, 6, 5], [0, 3, 7, 4]
]
var body: some View {
GeometryReader { geometry in
let projectedVertices = computeProjectedVertices(geometry.size)
Path { path in
// Draw faces
for face in faces {
path.move(to: projectedVertices[face[0]])
for i in 1..<face.count {
path.addLine(to: projectedVertices[face[i]])
}
path.closeSubpath()
}
}
.fill(Color.blue.opacity(0.5))
.overlay(
Path { path in
// Draw edges
for edge in edges {
let start = projectedVertices[edge[0]]
let end = projectedVertices[edge[1]]
path.move(to: start)
path.addLine(to: end)
}
}
.stroke(Color.blue, lineWidth: 2)
)
}
.frame(width: 500, height: 500)
.background(Color.gray.opacity(0.1))
.onAppear {
let _ = Timer.scheduledTimer(withTimeInterval: 0.016, repeats: true) { _ in
angleX += 0.01
angleY += 0.01
}
}
}
func computeProjectedVertices(_ size: CGSize) -> [CGPoint] {
let width = size.width
let height = size.height
let halfWidth = width / 2
let halfHeight = height / 2
return vertices.map { v -> CGPoint in
var (x, y, z) = (v[0], v[1], v[2])
// Rotate around X axis
let tempY = y * cos(angleX) - z * sin(angleX)
let tempZ = y * sin(angleX) + z * cos(angleX)
y = tempY
z = tempZ
// Rotate around Y axis
let tempX = x * cos(angleY) - z * sin(angleY)
z = x * sin(angleY) + z * cos(angleY)
x = tempX
// Project the 3D coordinates to 2D
let scale = 150 / (z + 4)
let projX = x * scale + halfWidth
let projY = y * scale + halfHeight
return CGPoint(x: projX, y: projY)
}
}
}
struct RotatingCube_Previews: PreviewProvider {
static var previews: some View {
RotatingCube()
}
}
Verdict
The quality of the results (at q8) look very good. In addition, the speed is good enough to use it locally for development work, particularly for large project changes with many files that require a high context length and would become very expensive over time on paid models. I’ll do more tests soon and report back.