As a person you can only send them money. As a business you can initiate a direct debit which withdraws money. However you are attesting that they signed a direct debit agreement with you and provided their account number and agreed on the amount to pay.
This is the same as a credit card - you can charge any card with just the number and a couple of basic details, however if there's a complaint "I found these CC details on a random website" isn't accepted, you need to show the card holder agreed to the charge. If you don't provide the evidence the transaction is reversed.
That is usually not how credit cards work anymore. Sure, you can try to charge any card but if it is issued by a European bank it will very likely be denied and you will be asked to do a Strong Customer Authentication.
Same applies to SEPA direct debit. Here in Sweden most (all?) banks requires the customer to sign digitally before any direct debit mandate is created.
This is the same as a credit card - you can charge any card with just the number and a couple of basic details, however if there's a complaint "I found these CC details on a random website" isn't accepted, you need to show the card holder agreed to the charge. If you don't provide the evidence the transaction is reversed.