What’s Click on To Do? Meet Microsoft’s subsequent AI headliner for Home windows PCs

Click on To Do allows you to feed textual content to AI fashions

Click on To Do makes use of optical character recognition (OCR) expertise to scan your present display screen and make textual content selectable. Mainly, it’s taking a screenshot and letting you work together with components inside it.

For instance, if you choose an e mail tackle, the Click on To Do menu presents “Ship e mail” to compose an e mail. If you choose an internet site URL, you’ll be able to select “Open web site” to launch it in your default net browser. (Fortunately, this doesn’t at all times use Microsoft Edge!)

If you choose over 10 phrases of textual content, issues get extra attention-grabbing. You’ll get a wide range of actions that use the Phi Silica language mannequin that runs in your Copilot+ PC’s NPU to carry out actions like summarizing textual content, making a bulleted listing, or rewriting the textual content in numerous tones.

That is one in every of Microsoft’s first makes an attempt at integrating NPU-powered textual content actions on a Copilot+ PC. As a result of it’s primarily based on a screenshot, you’ll be able to solely ship a lot textual content to the language mannequin directly.

That ought to enhance the expertise, as a result of let’s be sincere: these native language fashions that run in your Home windows PC are nowhere close to as highly effective as a cloud-based massive language fashions like ChatGPT. (Should you’re utilizing a chatbot app in your browser, you’ll surprise why you’re bothering to make use of a worse app that runs solely in your PC. That’s a core drawback with so many Copilot+ PC native AI options.)

Chris Hoffman / Foundry

There’s additionally an “Ask Copilot” motion that can ship your chosen textual content to Microsoft’s Copilot AI chatbot, and a “Draft with Copilot in Phrase” motion to begin engaged on a Phrase doc with Microsoft’s AI.

You may ship info to a chatbot within the cloud right here, too—however provided that you’re utilizing Microsoft’s Copilot (for house customers) or Microsoft 365 Copilot (for companies). Yeah, I do know, it may be complicated.