I've recently been looking into editing tools for my photography editing workflow. I've been using CaptureOne ever since the demise of Apple's Aperture 3, and complimenting this with Nik Collection (for removing noise, sharpening, some infrared processing and B&W) and Helicon Focus (for macro focus stacking). Recently Nik Collection apps stopped working on my Mac and it was time to invest either in the latest version or switch. I'd heard positive things in forums about Topaz sharpening and noise removal apps, so decided to download the free trials of both alongside the latest Nik Collection (v4 since you ask).

I have to say I was very impressed by Topaz Sharpen AI and Topaz Denoise AI. Both did their jobs exceptionally well and seemed to overlap one anothers' functionality quite nicely. I've always struggled with which to do first - removing noise and applying sharpening seem very much the same activity, you don't want to remove noise and lose detail that then can't be sharpened, while you also don't want to sharpen any noise. So it's great to see Denoise AI have some sharpening tools and Sharpen AI have some noise reduction tools. I was also blown away at how Topaz managed to find intense details in areas that no other app seemed able to. And that's when it occurred to me what these tools - and especially the Sharpen AI tool were probably doing.

Hands up, I'm no AI or ML expert. I'm hardly even a novice. But I do understand how you need to train such tools. The basic premise is to feed into the machine many photos to teach it how to identify things. Potentially for these tools, both pin sharp and blurred versions are used to help train the tool how to move from a blurry image to one that's sharp. So my assumption is blurry images I feed in are being sharpened as much as possible from the detail that does exist, and then further enhanced by effectively being compared to a huge index of other images and identifying similar patterns. Where such patterns are found, the sharpened details can be applied. This is no doubt on a very macro level, rather than finding identically composed images, but fundamentally this would mean that such tools are not sharpening details that exist in my image, but guessing what the detail would be if I had captured it. A very good example of this is in this video from Dave Kelly looking at butterfly wing detail being 'recovered'. I can't see how that fine wing detail (at 300% magnification) is there in the original on the left:

DaveKellyTopaz (with text)

So if indeed I'm correct, and new information is being added to the image, then this creates some really challenging questions for photographers. Is a photo sharpened in such tools still your image? Is it still a faithful representation of what was in front of the camera, or is it an approximation based on what similar images look like? While photoshop allows entire backgrounds to be swapped out, this seems to be doing similar both on an extreme micro level, a matter of pixels at a time. There are already examples of photography competitions specifically banning the use of AI/ML editing tools (discussed here). 

But what about others apps that use AI and ML in their workflow, such as masking tools that can identify subjects? And the in camera systems that rely on AI and ML, such as subject / eye recognition for focusing? I feel there is a distinction though. These examples are technologies that help improve your ability to capture and edit an image, but crucially the photographer is still in the driving seat. They are tools that enable you to capture the scene in front of you in pixel form and then edit those specific pixels. But where an area of pixels making up a blurry area of an image are being replaced with new pixels to create what the computer expects to be there then there's a difference. 

Nicely side stepping the the broader question of what IS photography (is it about representing what you saw, representing what you want the viewer to see, or something else?), I feel like the first step has to be transparency on how such tools are working and what is going on under the bonnet. Photographers can then decide for themselves whether or not to use such tools. But currently, I don't believe there is any information. I couldn't find any. It also meant it was very hard to sense check my above thinking because there didn't seem to be any other content on this topic (but please do let me know if/where I'm wrong above and I'll add edits and amendments).