Personal build issue when setting team.scm.deleteDestinationBeforeFetch to false?

My case is really simple, one component, with three change sets (A, B, C), each change set introduces a new file, test1.txt test2.txt test3.txt in order, workspace A contains all the change sets, workspace B contains only change set A, request build 1 with workspace A, and then request build 2 with workspace B, now, i suppose only test1.txt exist in the local sandbox on build machine, but the truth is, test2.txt and test3.txt are there too.

Depends on what are the roots being loaded. I am assuming you are not using loadrules to restrict what is loaded. If your component looks like:

/MyTopFolder
/MyTopFolder/test1.txt
/MyTopFolder/test2.txt
/MyTopFolder/test3.txt
and you don't have any load rules to restrict what is being loaded, then /MyTopFolder is what is loaded and test2.txt and test3.txt should not be there

If your component looks like
/test1.txt
/test2.txt
/test3.txt
And no load rules then all 3 files are top level items (shares) being loaded and test2.txt and test3.txt would remain since they are shares from a different workspace.

If you wanted to treat the component's root folder as the share, you would need to use a load rule and load it as the top level item.

My component looks like this,
/test1.txt
/test2.txt
/test3.txt
so is there any way to make sure that each personal build with different workspace just load the files the workspace have, no more, no less?

You would need to use a load rules. Specifically you need an ItemLoadRule to load the component root directory. See https://jazz.net/library/article/1015