I think is an Algernon Blackwood story but its not The Willows. The story i mean is in "Best Ghost Stories of Algernos Blackwood" The plot is exactly as described but i dont remember the title. I will look for it and post it.