【Python】マルチプロセス + ノンブロッキングI/O処理

1 min read読了の目安(約1100字

とりあえずコードを見てくれ。そして興味があれば調べてみるといい。

import asyncio
import time
import random
import concurrent.futures

# ランダム処理
def print_num(text):
    i = random.randrange(10)
    print (text + " - %s sec - 開始" % i)
    time.sleep(i)
    print (text + " 終了")

async def multi_process(loop,stock):
    executor = concurrent.futures.ProcessPoolExecutor()
    queue = asyncio.Queue()
    [queue.put_nowait(x) for x in stock]
    async def p(q):
        while not q.empty():
            i = await q.get()
            future = loop.run_in_executor(executor, print_num, i)
	    await future

    # 8プロセスで処理
    tasks = [asyncio.create_task(p(queue)) for i in range(8)]
    return await asyncio.wait(tasks)

def main():
    stock = ["aaa","bbb","ccc","ddd","eee","fff","ggg","hhh"]
    loop = asyncio.get_event_loop()
    loop.run_until_complete(multi_process(loop,stock))

if __name__ == "__main__":  main()

これを使用したツール
Github

https://github.com/Trusted-Dream/Media-Downloader_for_LINE/blob/7cad554a992837984cf962b67fa215bce878c54a/lib/downloader.py#L80

動作状況をYoutubeで

https://www.youtube.com/watch?v=NHd-SMjSdNE