Qwen Coder 2.5 32b on M4 Max

Published: 2024-11-11 10:30:30

Initial notes on how Qwen Coder 2.5 performs on a Macbook Pro M4 Max

I received my new Macbook Pro M4 Max today and at the same time Qwen released their new Coder 2.5 model.

I quickly installed LMStudio, downloaded the GGUF models and ran some tests.

I used two different GGUF models for these tests:

  • Qwen Coder 2.5 32b q8
  • Qwen Coder 2.5 32b q2_k

I chose these two to observer the difference in performance between lightly and heavily quantized models.

Environment

Those were loaded with the following settings:

  • Context Length: 4096k
  • Flash Attention: Disabled
  • GPU Offload: Full
  • CPU Thread Pool Size: 12
  • Evaluation Batch Size 512
  • Keep Model in Memory: True
  • Try mmap: True

First Prompt

Results on this prompt (48 tokens):

I have this react fn in typescript

export function LoginForm()

how can I change it so that it can receive these two functions as parameters:

export async function signup(formData: FormData) {}

export async function login(formData: FormData) {}

(Note that the prompt is deliberately not well written)

q8

  • Response: 12.96 tok/sec, 564 tokens, 1.02s to first token
Response
To modify your `LoginForm` component to accept the `signup` and `login` functions as parameters, you'll need to define an interface for the props that the component will receive. This ensures type safety in TypeScript.

Here's how you can do it:

  1. Define a LoginFormProps interface that includes the types for the signup and login functions.
  2. Use these props within your LoginForm component.

Here is the updated code:

import React from 'react';

// Define the interface for the LoginForm props
interface LoginFormProps {
  signup: (formData: FormData) => Promise<void>;
  login: (formData: FormData) => Promise<void>;
}

export function LoginForm({ signup, login }: LoginFormProps) {

  // Example usage of the signup and login functions within the component
  const handleSubmit = async (event: React.FormEvent<HTMLFormElement>) => {
    event.preventDefault();
    const formData = new FormData(event.currentTarget);

    // Determine if it's a signup or login attempt
    // This is just an example, you might want to use another mechanism to determine this.
    const isSignupAttempt = true; // Change this based on your logic

    try {
      if (isSignupAttempt) {
        await signup(formData);
      } else {
        await login(formData);
      }
    } catch (error) {
      console.error('Error during form submission:', error);
    }
  };

  return (
    <form onSubmit={handleSubmit}>
      {/* Your form fields go here */}
      <button type="submit">Submit</button>
    </form>
  );
}

// Example usage of LoginForm component
export async function signup(formData: FormData) {
  // Signup logic here
  console.log('Signing up with:', formData);
}

export async function login(formData: FormData) {
  // Login logic here
  console.log('Logging in with:', formData);
}

In this example:

  • LoginFormProps defines the shape of the props that LoginForm will receive, including the types for the signup and login functions.
  • The handleSubmit function is an example of how you might use these functions within the component. It determines whether to call signup or login based on some condition (isSignupAttempt) which you should adjust according to your specific logic.
  • The form element uses the onSubmit event handler to trigger the handleSubmit function when the form is submitted.

You can now pass the signup and login functions as props when rendering the LoginForm component:

<LoginForm signup={signup} login={login} />

This ensures that your component is flexible and can handle different implementations of the signup and login functions.

q2_k
  • Response: 17.95 tok/sec, 494 tokens, 0.76s to first token
Response
To modify your `LoginForm` component to accept the `signup` and `login` functions as parameters, you need to define a TypeScript interface or type for the props that your component will receive. This allows you to specify that the component should expect these two functions as part of its props.

Here's how you can do it:

  1. Define an interface for the props.
  2. Use this interface in the LoginForm function definition.

Here is the updated code:

import React from 'react';

// Define the type for the props that LoginForm will receive
interface LoginFormProps {
  signup: (formData: FormData) => Promise<void>;
  login: (formData: FormData) => Promise<void>;
}

export function LoginForm({ signup, login }: LoginFormProps) {
  // Your form rendering logic here

  const handleSubmit = async (event: React.FormEvent<HTMLFormElement>) => {
    event.preventDefault();
    const formData = new FormData(event.currentTarget);
    
    // Example of calling the passed functions
    if (someCondition) { // Replace with your actual condition to decide between signup and login
      await signup(formData);
    } else {
      await login(formData);
    }
  };

  return (
    <form onSubmit={handleSubmit}>
      {/* Your form fields here */}
      <button type="submit">Submit</button>
    </form>
  );
}

// Example usage of LoginForm
// const App = () => {
//   const handleSignup = async (formData: FormData) => {
//     // signup logic
//   };

//   const handleLogin = async (formData: FormData) => {
//     // login logic
//   };

//   return <LoginForm signup={handleSignup} login={handleLogin} />;
// };

In this example:

  • LoginFormProps is an interface that defines the shape of the props expected by LoginForm. It includes two properties: signup and login, each being a function that takes a FormData object and returns a Promise<void>.
  • The LoginForm component deconstructs the props to receive signup and login directly.
  • A sample form handling logic is included, where you can determine which function (signup or login) should be called based on some condition.

You can then pass the signup and login functions when using the LoginForm component, as shown in the commented example usage.

Quality

If you open the responses, you'll see that the quality at q2_k really degrades. At the same time, the time difference of 12 to 17 seconds is not that long.

Second Prompt

Please find bugs in the following typescript code and explain in detail what the bug is and provide a git patch with a fix:

import React from 'react';

// Define the interface for the LoginForm props
interface LoginFormProps {
  signup: (formData: FormData) => Promise<void>;
  login: (formData: FormData) => Promise<void>;
}

export function LoginForm({ signup, login }: LoginFormProps) {

  // Example usage of the signup and login functions within the component
  const handleSubmit = async (event: React.FormEvent<HTMLFormElement>) => {
    event.preventDefault();
    const formData = new FormData(event.currentTarget);

    // Determine if it's a signup or login attempt
    // This is just an example, you might want to use another mechanism to determine this.
    const isSignupAttempt = true; // Change this based on your logic

    try {
      if (isSignupAttempt) {
        await signup(formData);
      } else {
        await signup(formData);
      }
    } catch (error) {
      console.error('Error during form submission:', error);
    }
  };

  return (
    <form onSubmit={handleSubmit}>
      {/* Your form fields go here */}
      <button type="submitt">Submit</button>
    </form>
  );
}

// Example usage of LoginForm component
export async function signup(formData: FormData) {
  // Signup logic here
  console.log('Signing up with:', formData);
}

export async function login(formData: FormData) {
  // Login logic here
  console.log('Logging in with:', formData);
}

Results

q8

  • Response: 12.72 tok/sec, 536 tokens, 2.20s to first token
Response
### Bugs Identified:
  1. Incorrect Function Call:

    • In the handleSubmit function, when isSignupAttempt is false, it incorrectly calls signup(formData) instead of login(formData).
  2. Typo in Button Type Attribute:

    • The <button> element has a typo in its type attribute: submitt should be submit.

Detailed Explanation:

  1. Incorrect Function Call:

    • This bug will cause the application to always attempt to sign up, even if the intent is to log in. When isSignupAttempt is false, the code should call the login function instead of signup.
  2. Typo in Button Type Attribute:

    • The typo in the button type means that clicking the button will not trigger the form's onSubmit event handler as intended. The correct value for a submit button is submit.

Git Patch with Fixes:

Here is the patch file (0001-Fix-signup-login-call-and-typo-in-button-type.patch) to address these issues.

diff --git a/src/LoginForm.tsx b/src/LoginForm.tsx
index 1234567..89abcde 100644
--- a/src/LoginForm.tsx
+++ b/src/LoginForm.tsx
@@ -20,7 +20,7 @@ export function LoginForm({ signup, login }: LoginFormProps) {
       if (isSignupAttempt) {
         await signup(formData);
       } else {
-        await signup(formData);
+        await login(formData);
       }
     } catch (error) {
       console.error('Error during form submission:', error);
@@ -34,7 +34,7 @@ export function LoginForm({ signup, login }: LoginFormProps) {
     </form>
   );
 }
-
+
 // Example usage of LoginForm component
 export async function signup(formData: FormData) {
   // Signup logic here

Applying the Patch:

  1. Save the above content into a file named 0001-Fix-signup-login-call-and-typo-in-button-type.patch.
  2. Navigate to your project directory in the terminal.
  3. Apply the patch using the git apply command:
    git apply 0001-Fix-signup-login-call-and-typo-in-button-type.patch
    

After applying the patch, the code should work correctly by calling the appropriate function based on the value of isSignupAttempt, and the button will properly submit the form.

q2_k

  • Response: 17.95 tok/sec, 453 tokens, 2.08s to first token
Response
The provided TypeScript code contains a few issues that need to be addressed. Let's go through each bug and provide the necessary fixes.

Bugs Identified:

  1. Incorrect Function Call: The handleSubmit function incorrectly calls the signup function twice instead of calling login for login attempts.
  2. Typo in Button Type Attribute: The button type is misspelled as "submitt" instead of "submit".

Let's address these issues with a git patch.

Git Patch

diff --git a/src/LoginForm.tsx b/src/LoginForm.tsx
index 1234567..89abcde 100644
--- a/src/LoginForm.tsx
+++ b/src/LoginForm.tsx
@@ -18,7 +18,7 @@ export function LoginForm({ signup, login }: LoginFormProps) {
     try {
       if (isSignupAttempt) {
         await signup(formData);
-      } else {
-        await signup(formData);
+      } else {
+        await login(formData);
       }
     } catch (error) {
       console.error('Error during form submission:', error);
@@ -30,7 +30,7 @@ export function LoginForm({ signup, login }: LoginFormProps) {
       {/* Your form fields go here */}
-      <button type="submitt">Submit</button>
+      <button type="submit">Submit</button>
     </form>
   );
 }

Explanation of Fixes:

  1. Correct Function Call: The handleSubmit function now correctly calls the login function when it is not a signup attempt.
    -        await signup(formData);
    +        await login(formData);
    
  2. Button Type Correction: The button type attribute was corrected from "submitt" to "submit".
    -      <button type="submitt">Submit</button>
    +      <button type="submit">Submit</button>
    

By applying this patch, the LoginForm component should work as intended, calling either the signup or login function based on the form submission context and correctly handling button submissions.

Parameters

I also wanted to try the same prompt with different model settings. So here the prompt is as follows:

Can you explain what this code does and then convert it to swiftui?

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>3D Rotating Cube</title>
    <style>
        body {
            display: flex;
            justify-content: center;
            align-items: center;
            height: 100vh;
            margin: 0;
            background-color: #f0f0f0;
        }
        canvas {
            background-color: #ffffff;
        }
    </style>
</head>
<body>
    <canvas id="cubeCanvas" width="500" height="500"></canvas>
    <script>
        const canvas = document.getElementById("cubeCanvas");
        const ctx = canvas.getContext("2d");
        const width = canvas.width;
        const height = canvas.height;
        const halfWidth = width / 2;
        const halfHeight = height / 2;

        let angleX = 0;
        let angleY = 0;

        function drawCube() {
            ctx.clearRect(0, 0, width, height);

            const vertices = [
                [-1, -1, -1], [1, -1, -1], [1, 1, -1], [-1, 1, -1],
                [-1, -1, 1], [1, -1, 1], [1, 1, 1], [-1, 1, 1]
            ];

            const edges = [
                [0, 1], [1, 2], [2, 3], [3, 0],
                [4, 5], [5, 6], [6, 7], [7, 4],
                [0, 4], [1, 5], [2, 6], [3, 7]
            ];

            const faces = [
                [0, 1, 2, 3], [4, 5, 6, 7], [0, 1, 5, 4],
                [2, 3, 7, 6], [1, 2, 6, 5], [0, 3, 7, 4]
            ];

            const projectedVertices = vertices.map(v => {
                let [x, y, z] = v;

                // Rotate around X axis
                let tempY = y * Math.cos(angleX) - z * Math.sin(angleX);
                let tempZ = y * Math.sin(angleX) + z * Math.cos(angleX);
                y = tempY;
                z = tempZ;

                // Rotate around Y axis
                let tempX = x * Math.cos(angleY) - z * Math.sin(angleY);
                z = x * Math.sin(angleY) + z * Math.cos(angleY);
                x = tempX;

                // Project the 3D coordinates to 2D
                const scale = 200 / (z + 4);
                const projX = x * scale + halfWidth;
                const projY = y * scale + halfHeight;
                return [projX, projY];
            });

            // Draw faces
            ctx.fillStyle = "rgba(173, 216, 230, 0.5)"; // light blue
            faces.forEach(face => {
                ctx.beginPath();
                ctx.moveTo(...projectedVertices[face[0]]);
                for (let i = 1; i < face.length; i++) {
                    ctx.lineTo(...projectedVertices[face[i]]);
                }
                ctx.closePath();
                ctx.fill();
            });

            // Draw edges
            ctx.strokeStyle = "darkblue";
            ctx.lineWidth = 2;
            edges.forEach(edge => {
                const [start, end] = edge;
                ctx.beginPath();
                ctx.moveTo(...projectedVertices[start]);
                ctx.lineTo(...projectedVertices[end]);
                ctx.stroke();
            });
        }

        function animate() {
            angleX += 0.01;
            angleY += 0.01;
            drawCube();
            requestAnimationFrame(animate);
        }

        animate();
    </script>
</body>
</html>

Results

q2_k

  • ~8k context, Enable Flash Attention
  • 16.92 tok/sec, 859 tokens, 5.45s to first token
  • Result didn't compile in Xcode, has 4 errors which are probably easy to fix
Response
import SwiftUI import SceneKit

struct ContentView: View { @State private var angleX: Float = 0.0 @State private var angleY: Float = 0.0

var body: some View {
    VStack {
        SCNVeviewContainer(angleX: $angleX, angleY: $angleY)
            .frame(width: 500, height: 500)
        Spacer()
    }
    .onAppear {
        let _ = Timer.scheduling(timerTick, interval: 0.016) // Approximately 60 FPS
    }
}

func timerTick(_ timer: Timer) {
    angleX += 0.01
    angleY += 0.01
}

}

struct SCNVeviewContainer: UIViewRepresentable { @Binding var angleX: Float @Binding var angleY: Float

let sceneView = SCNView()

func makeUIView(context: Context) -> SCNView {
    // Set the background color of the scene view
    sceneView.backgroundColor = UIColor.white
    
    // Create a new SCNScene
    let scene = SCNScene()
    
    // Create a cube geometry
    let cubeGeometry = SCNBox(width: 1.0, height: 1.0, length: 1.0, chamferRadius: 0)
    cubeGeometry.firstMaterial?.diffuse.contents = UIColor(red: 0.68, green: 0.85, blue: 0.92, alpha: 0.5) // Light blue with transparency
    
    // Create a node and add the geometry to it
    let node = SCNNode(geometry: cubeGeometry)
    
    // Set the initial position of the node
    node.position = SCNVector3(0, 0, -4)
    
    // Add the node to the scene
    scene.rootNode.addChildNode(node)
    
    // Set the scene to the view
    sceneView.scene = scene
    
    return sceneView
}

func updateUIView(_ uiView: SCNView, context: Context) {
    let rotationX = SCNRotation(x: 1, y: 0, z: 0, angle: angleX)
    let rotationY = SCNRotation(x: 0, y: 1, z: 0, angle: angleY)
    
    // Apply the rotations to the node
    uiView.scene!.rootNode.enumerateChildNodes { (node, stop) in
        node.rotation = rotationX * rotationY
    }
}

}

@main struct ContentViewApp: App { var body: some Scene { WindowGroup { ContentView() .frame(width: 500, height: 500) .background(Color(.lightGray)) } } }

q2_k

  • full context, Disable Flash Attention
  • 15.22 tok/sec, 990 tokens, 6.95s to first token
  • Result didn't compile in Xcode, has 1 error (wrong pattern matching) which should be easy to fix
Response
import SwiftUI

struct ContentView: View { @State private var angleX: Double = 0 @State private var angleY: Double = 0

let vertices: [[Double]] = [
    [-1, -1, -1], [1, -1, -1], [1, 1, -1], [-1, 1, -1],
    [-1, -1, 1], [1, -1, 1], [1, 1, 1], [-1, 1, 1]
]

let edges: [[Int]] = [
    [0, 1], [1, 2], [2, 3], [3, 0],
    [4, 5], [5, 6], [6, 7], [7, 4],
    [0, 4], [1, 5], [2, 6], [3, 7]
]

let faces: [[Int]] = [
    [0, 1, 2, 3], [4, 5, 6, 7], [0, 1, 5, 4],
    [2, 3, 7, 6], [1, 2, 6, 5], [0, 3, 7, 4]
]

var body: some View {
    GeometryReader { geometry in
        Path { path in
            let width = geometry.size.width
            let height = geometry.size.height
            let halfWidth = width / 2
            let halfHeight = height / 2
            
            let projectedVertices = vertices.map { v in
                let [x, y, z] = v
                
                // Rotate around X axis
                let tempY = y * cos(angleX) - z * sin(angleX)
                let tempZ = y * sin(angleX) + z * cos(angleX)
                
                // Rotate around Y axis
                let tempX = x * cos(angleY) - tempZ * sin(angleY)
                let finalZ = x * sin(angleY) + tempZ * cos(angleY)
                
                // Project the 3D coordinates to 2D
                let scale = 200 / (finalZ + 4)
                let projX = tempX * scale + halfWidth
                let projY = tempY * scale + halfHeight
                
                return CGPoint(x: projX, y: projY)
            }
            
            // Draw faces
            path.setFillColor(Color(red: 173/255, green: 216/255, blue: 230/255, opacity: 0.5))
            for face in faces {
                let points = face.map { projectedVertices[$0] }
                path.addLines(points)
                path.closeSubpath()
            }
            
            // Draw edges
            path.setStrokeColor(Color.blue)
            path.setLineWidth(2)
            for edge in edges {
                path.move(to: projectedVertices[edge[0]])
                path.addLine(to: projectedVertices[edge[1]])
            }
        }
    }
    .onAppear {
        Timer.scheduledTimer(withTimeInterval: 0.05, repeats: true) { _ in
            self.angleX += 0.02
            self.angleY += 0.02
        }
    }
    .background(Color.gray)
}

}

struct ContentView_Previews: PreviewProvider { static var previews: some View { ContentView() } }

q8

  • No flash attention, 4k context
  • 9.77 tok/sec, 1813 tokens, 5.58 to first token
  • The result didn't compile in Xcode, but I asked the model for the errors and it helped me fix them in another two iterations.
import SwiftUI

struct RotatingCube: View {
    @State private var angleX = 0.0
    @State private var angleY = 0.0
    
    let vertices: [[Double]] = [
        [-1, -1, -1], [1, -1, -1], [1, 1, -1], [-1, 1, -1],
        [-1, -1, 1], [1, -1, 1], [1, 1, 1], [-1, 1, 1]
    ]
    
    let edges: [[Int]] = [
        [0, 1], [1, 2], [2, 3], [3, 0],
        [4, 5], [5, 6], [6, 7], [7, 4],
        [0, 4], [1, 5], [2, 6], [3, 7]
    ]
    
    let faces: [[Int]] = [
        [0, 1, 2, 3], [4, 5, 6, 7], [0, 1, 5, 4],
        [2, 3, 7, 6], [1, 2, 6, 5], [0, 3, 7, 4]
    ]
    
    var body: some View {
        GeometryReader { geometry in
            let projectedVertices = computeProjectedVertices(geometry.size)
            
            Path { path in
                // Draw faces
                for face in faces {
                    path.move(to: projectedVertices[face[0]])
                    for i in 1..<face.count {
                        path.addLine(to: projectedVertices[face[i]])
                    }
                    path.closeSubpath()
                }
            }
            .fill(Color.blue.opacity(0.5))
            .overlay(
                Path { path in
                    // Draw edges
                    for edge in edges {
                        let start = projectedVertices[edge[0]]
                        let end = projectedVertices[edge[1]]
                        path.move(to: start)
                        path.addLine(to: end)
                    }
                }
                    .stroke(Color.blue, lineWidth: 2)
            )
        }
        .frame(width: 500, height: 500)
        .background(Color.gray.opacity(0.1))
        .onAppear {
            let _ = Timer.scheduledTimer(withTimeInterval: 0.016, repeats: true) { _ in
                angleX += 0.01
                angleY += 0.01
            }
        }
    }
    
    func computeProjectedVertices(_ size: CGSize) -> [CGPoint] {
        let width = size.width
        let height = size.height
        let halfWidth = width / 2
        let halfHeight = height / 2
        
        return vertices.map { v -> CGPoint in
            var (x, y, z) = (v[0], v[1], v[2])
            
            // Rotate around X axis
            let tempY = y * cos(angleX) - z * sin(angleX)
            let tempZ = y * sin(angleX) + z * cos(angleX)
            y = tempY
            z = tempZ
            
            // Rotate around Y axis
            let tempX = x * cos(angleY) - z * sin(angleY)
            z = x * sin(angleY) + z * cos(angleY)
            x = tempX
            
            // Project the 3D coordinates to 2D
            let scale = 150 / (z + 4)
            let projX = x * scale + halfWidth
            let projY = y * scale + halfHeight
            return CGPoint(x: projX, y: projY)
        }
    }
}

struct RotatingCube_Previews: PreviewProvider {
    static var previews: some View {
        RotatingCube()
    }
}

Animated

Verdict

The quality of the results (at q8) look very good. In addition, the speed is good enough to use it locally for development work, particularly for large project changes with many files that require a high context length and would become very expensive over time on paid models. I'll do more tests soon and report back.