Actually, a person should be very careful about sending a letter acknowledging the situation and affirming a denial of permission to use the property.
There is a doctrine under the common law and many state’s statutes called
adverse possession that allows a person to take your land over your objection if certain criteria are met. One of those criteria is openly taking possession of the land in a way that ought to make you aware that the seizure of the land has happened. If the person takes your land and you do not sue the person within the statute of limitations, but are or should have been aware of the seizure, then the land may become theirs. Sending a letter telling them ‘I don’t give you permission to do this”, but not following through with a suit, may simply function as evidence the neighbor can use against you to show you knew about the taking and did nothing.
https://www.nolo.com/legal-encyclopedia/adverse-possession-trespassers-become-owners-46934.html