Tight defence had ever helped Japan caught up in the final period, including a half-court trap against Lin Chih-cheih that led to a Sakurai Ryota layup on the other end. But Lin fought back, driving to the hoop to keep the lead at 77-73 with 44 seconds remaining.
Japan called for timeout, but still could not come closer. Tsuji Naoto missed a three-pointer and had to resort to intentional fouls to preserve the clock.
Chinese Taipei's Tseng Wen-ting made both tries from the charity stripe with 25 seconds left.
With 10 seconds left, Tsuji tried another triple and was fouled from beyond the arc. He made three free throws, but Chinese Taipei called timeout to inbound the ball safely, holding on to the win.
Tseng led Chinese Taipei with 19 points while Lin had 16 and six rebounds. Tien Lei and Chen Shih-chieh added 10 points apiece.
Chinese Taipei are topping Group E. If Qatar defeat hosts Philippines later today, Qatar and Chinese Taipei will play for the first place in the group on Wednesday.
Japan falls to 1-3, but will still likely advance to the quarterfinals if they defeat Jordan in their final group match.