Drag-Drop improvements

[amwx]

Is your feature request related to a problem? Please describe.

A couple issues I've come across with drag-drop (in-app, not to/from external process):

1 - IMO, the drag-drop events really need to sit in one of the base classes to all controls like it is in WPF/UWP (in UIElement). There are some cases where being able to do some logic before or after one of the drag events is raised is super helpful (thus needing a protected virtual method) as you can't rely on AddHandler() to do, particularly in the latter case.

2 - If a control that acts as a drag source contains a ToolTip, the ToolTip needs to be hidden as part of drag drop activating, which follows WPF. Otherwise, the ToolTip will hide, but the drag/drop icon flickers as the pointer moves.

3 - DragLeave event should have DragEventArgs instead of just RoutedEventArgs - so pointer information can be retrieved

4 - One of the things I also was trying to implement, similar to UWP dragging, is a popup with a preview of the item being dragged - except it seems the popup doesn't respond to the changes in position during drag drop. popup.Host.ConfigurePosition() is called but the popup remains in one place. It also seems that all of the InputManager methods (process, preprocess, postprocess) don't do anything during drag drop and return (-1,-1) only once upon querying RawPointerEventArgs.Position, which makes this task impossible to achieve in any way that I can see. I've tested with WPF and this is possible

5 Current implementation preprocess events in global static IInputManager to simulate dnd

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

[robloo]

It might help to emphasize these issues come from real-world use cases -- my guess when implementing/testing the TabView port.

[out4blood88]

For 4. above, my use case is: the user has to be able to drag and drop something, and once they start dragging it, a visual copy of the element follows the user's cursor around until they drop it (see tab dragging in Firefox for an example). Generating the visual copy is easy enough, but when attempting to move it to follow the user's cursor, I too ran into the issue where subscribing to the raw events didn't work:

IInputManager inputManager = AvaloniaLocator.Current.GetService<IInputManager>()!;
inputManager.PostProcess.OfType<RawPointerEventArgs>().Subscribe(OnPointerMovedRaw);

OnPointerMovedRaw will get called only once after dragging starts, and the position in it is -1,-1.

On Windows, I believe this thread about the same problem in WPF highlights the challenge:

Digging deeper (through the ole2.dll source code), I discovered that a separate, invisible window gets created during the drag which eats up all the normal messages and instead interfaces with the drop targets directly (which is probably why the normal WPF mouse events don't fire in the first place).

I successfully used the workaround as highlighted here, again for WPF:

I tried your demo, and found that if you set AllowDrop="True" on the window, the DragOver event works.

Setting AllowDrop="true" on the window does give you DragOver events as you would hope for. Of course, this makes it so that DragEnter and DragLeave are essentially meaningless (although you still do need to handle setting the DragEffects to DragDropEffects.None to show the user that they can't drop it anywhere in the window).

Then you would determine manually if the pointer is over your actual drop targets like this:

var isLeftBoxHit = _leftBox.Bounds.Contains(e.GetPosition(_panel));
if (isLeftBoxHit)
{
    Debug.WriteLine("Hit left");
}

Definitely hacky, but it does work.

[out4blood88]

I'm now using the approach referenced here, and with a full reference implementation here.

The hooked mouse events are sent back into the IInputManager so that subscribing to the IInputManager like AvaloniaLocator.Current.GetService<IInputManager>()!.PreProcess.OfType<RawPointerEventArgs>().Subscribe(OnPointerMovedRaw) will still receive mouse movement events as expected during a drag:

private async void DoDrag(object? sender, PointerPressedEventArgs e)
{
    var inputManager = AvaloniaLocator.Current.GetService<IInputManager>()!;
    var mouse = new MouseDevice();
    using var hookId = NativeMethods.HookMouseMove(pixelPoint =>
    {
        var timestamp = new DateTimeOffset(DateTime.UtcNow).ToUnixTimeSeconds();
        var point = this.PointToClient(pixelPoint);
        var eventArgs = new RawPointerEventArgs(mouse, (ulong)timestamp, this, RawPointerEventType.Move, point,
            RawInputModifiers.None);
        inputManager.ProcessInput(eventArgs);
    });

    var dragData = new DataObject();
    var result = await DragDrop.DoDragDrop(e, dragData, DragDropEffects.Link);
}

internal static class NativeMethods
{
    private delegate IntPtr LowLevelMouseProc(int nCode, IntPtr wParam, IntPtr lParam);

    private const int WH_MOUSE_LL = 14;

    private static IntPtr _hookID = IntPtr.Zero;
    private static Action<PixelPoint> _mouseMoveHandler;

    // wee need to keep the variable to prevent the GarbageCollector to remove the HookCallback
    // https://social.msdn.microsoft.com/Forums/vstudio/en-US/68fdc3dc-8d77-48c4-875c-5312baa56aee/how-to-fix-callbackoncollecteddelegate-exception?forum=netfxbcl
    private static LowLevelMouseProc _proc = HookCallback;

    internal static IDisposable HookMouseMove(Action<PixelPoint> mouseMoveHandler)
    {
        _mouseMoveHandler = mouseMoveHandler;

        using var process = Process.GetCurrentProcess();
        using var module = process.MainModule;
        var hhk = SetWindowsHookEx(WH_MOUSE_LL, _proc, GetModuleHandle(module.ModuleName), 0);
        return new HookedMouseMoveContext(hhk);
    }

    internal static void RemoveHook(IntPtr hookId)
    {
        UnhookWindowsHookEx(hookId);
    }

    private static IntPtr HookCallback(int nCode, IntPtr wParam, IntPtr lParam)
    {
        if (nCode >= 0 && MouseMessages.WM_MOUSEMOVE == (MouseMessages)wParam)
        {
            var hookStruct = Marshal.PtrToStructure<MSLLHOOKSTRUCT>(lParam);
            var point = new PixelPoint(hookStruct.pt.x, hookStruct.pt.y);
            _mouseMoveHandler(point);
        }
        return CallNextHookEx(_hookID, nCode, wParam, lParam);
    }

    private enum MouseMessages
    {
        WM_LBUTTONDOWN = 0x0201,
        WM_LBUTTONUP = 0x0202,
        WM_MOUSEMOVE = 0x0200,
        WM_MOUSEWHEEL = 0x020A,
        WM_RBUTTONDOWN = 0x0204,
        WM_RBUTTONUP = 0x0205
    }

    [StructLayout(LayoutKind.Sequential)]
    private struct POINT
    {
        public int x;
        public int y;
    }

    [StructLayout(LayoutKind.Sequential)]
    private struct MSLLHOOKSTRUCT
    {
        public POINT pt;
        public uint mouseData;
        public uint flags;
        public uint time;
        public IntPtr dwExtraInfo;
    }

    [DllImport("user32.dll", CharSet = CharSet.Auto, SetLastError = true)]
    private static extern IntPtr SetWindowsHookEx(int idHook, LowLevelMouseProc lpfn, IntPtr hMod, uint dwThreadId);

    [DllImport("user32.dll", CharSet = CharSet.Auto, SetLastError = true)]
    [return: MarshalAs(UnmanagedType.Bool)]
    private static extern bool UnhookWindowsHookEx(IntPtr hhk);

    [DllImport("user32.dll", CharSet = CharSet.Auto, SetLastError = true)]
    private static extern IntPtr CallNextHookEx(IntPtr hhk, int nCode, IntPtr wParam, IntPtr lParam);

    [DllImport("kernel32.dll", CharSet = CharSet.Auto, SetLastError = true)]
    private static extern IntPtr GetModuleHandle(string lpModuleName);
}

internal class HookedMouseMoveContext : IDisposable
{
    private readonly IntPtr _hhk;
    
    public HookedMouseMoveContext(IntPtr hhk)
    {
        _hhk = hhk;
    }
    
    public void Dispose()
    {
        NativeMethods.RemoveHook(_hhk);
    }
}

Is there any appetite to put this into Avalonia? Presumably it would look something like this, although I don't know the best way to get access to the appropriate IInputDevice and IVisualRoot.

class DragSource : IPlatformDragSource
{
    public unsafe Task<DragDropEffects> DoDragDrop(PointerEventArgs triggerEvent,
        IDataObject data, DragDropEffects allowedEffects)
    {
        Dispatcher.UIThread.VerifyAccess();

        triggerEvent.Pointer.Capture(null);
        
        using var dataObject = new DataObject(data);
        using var src = new OleDragSource();
        var allowed = OleDropTarget.ConvertDropEffect(allowedEffects);
        
        var objPtr = MicroComRuntime.GetNativeIntPtr<Win32Com.IDataObject>(dataObject);
        var srcPtr = MicroComRuntime.GetNativeIntPtr<Win32Com.IDropSource>(src);
        
        using var hookId = UnmanagedMethods.HookMouseMove(pixelPoint =>
        {
            var timestamp = new DateTimeOffset(DateTime.UtcNow).ToUnixTimeSeconds();
            var point = _visualRoot.PointToClient(pixelPoint);
            var eventArgs = new RawPointerEventArgs(_mouse, (ulong)timestamp, this, RawPointerEventType.Move, point,
                RawInputModifiers.None);
            _inputManager.ProcessInput(eventArgs);
        });

        UnmanagedMethods.DoDragDrop(objPtr, srcPtr, (int)allowed, out var finalEffect);
        
        // Force releasing of internal wrapper to avoid memory leak, if drop target keeps com reference.
        dataObject.ReleaseWrapped();

        return Task.FromResult(OleDropTarget.ConvertDropEffect((Win32Com.DropEffect)finalEffect));
    }
}

I'm happy to open a PR if this is approach is acceptable.

I have no idea who to reference here, apologies if this is a bit wide. @robloo @grokys @maxkatz6.

[amwx]

I've done some more investigating into this, particularly (4). The InputManager returning (-1,-1) is only occuring because the wrong RawEvent is being queried. If you use RawDragEvent instead of RawPointerEventArgs you can still get input messages during drag drop. The (-1,-1) is from a random WM_MOUSELEAVE event that occurs shortly after starting DragDrop However, this will only work while dragging inside the HWND. In WPF you can get this to work by handling the GiveFeedback event, originating from IOleDropSource.OleGiveFeedback but to get the mouse position outside the window you need GetCursorPos() which obviously doesn't work for touch/pen. Though I have no idea if the other platforms have something similar to OleGiveFeedback/is a way to mock it for parity. So (4) may not be entirely possible in a cross-platform, modern framework like Avalonia. Not to mention, I still cannot figure out why the popup refuses to move when in drag-drop (even if force-repositioned after RawDragEvent)

UWP does have CoreDragOperation.SetDragUIContentFromSoftwareBitmap, so there might be an option to pass in a bitmap to be set as the mouse cursor that gives the drag preview. Still requires the GiveFeedback event, has the downside of removing the default drag feedback cursors, and probably isn't cross-platform.

Also an aside, RawDragEvent probably should be renamed to RawDragEventArgs to be consistent with the other raw event args.

[Hanprogramer]

I think Avalonia needs a way to allow/not allow drag and drop from code. This would be better explained in a scenario:
You're trying to implement drag and drop, then there's a target to drop files, but it only accepts certain file formats, for example just PNGs, we need a way to handle this properly.

[tkefauver]

@amwx

2 - If a control that acts as a drag source contains a ToolTip, the ToolTip needs to be hidden as part of drag drop activating, which follows WPF. Otherwise, the ToolTip will hide, but the drag/drop icon flickers as the pointer moves.

Thank you!! I was about to give up but this clued me in to fixing my dnd!! Spent the last 2 ENTIRE days racking my brain because my dnd broke dragging into external applications. But only for 1 type of control and its because of a freakin' tooltip I added! (its only conditionally visible but was still directly attached to a parent of the drag source)

This situation does more than just make the cursor flicker (only testing on Win11 right now) it'll invalidate the whole operation. So to fix I moved the tooltip into a style so its only attached if not dragging. Thank you man