INDEX
2024-01-17 19:10 - Run redis as caching for nginx So the current thing I'm trying to do is run a basic array of containers: nginx, redis, website Here's the idea: basic port 80 nginx - basic website + redis caching - all separate containers systemctl start docker docker run -it --rm nginx bash # Open a shell inside nginx (see basic config) # Seems like if I put code in /usr/share/nginx/html the defaults work fine yum install docker-compose docker compose up -d # Run in background # Website should work now docker logs nginx-redis # Showing it works fine Now need to add redis container # Remember to add a "networks:" section at the bottom for any networks you use # Need to make all folders referenced in compose file docker compose down # Also kills networks # Even in a network, ports are exposed to my client. docker exec -it nginxredis-nginx-1 sh docker compose restart Now how do I add nginx support for redis Turns out to add support for the redis module you need to build it from source Guess I'm adding a dockerfile Now trying "docker compose build" to build this image - giving errors on config steps docker run -it --rm nginx:1-alpine sh # Diagnose issues by running manually # A few packages were missing so had to install those to build too # Dockerfile (doesn't work as gives a build error) FROM nginx:1-alpine RUN apk --no-cache add build-base pcre-dev zlib-dev RUN wget http://nginx.org/download/nginx-1.25.3.tar.gz && tar -zxvf nginx-1.25.3.tar.gz RUN wget https://people.freebsd.org/~osa/ngx_http_redis-0.3.9.tar.gz && \ tar -zxvf ngx_http_redis-0.3.9.tar.gz RUN cd nginx-1.25.3 && ./configure --add-module=/ngx_http_redis-0.3.9 && make && make install RUN rm -rf /nginx-1.25.3 && apk del build-base EXPOSE 80 CMD ["nginx", "-g", "daemon off;"] https://www.youtube.com/watch?v=gHgYA7u9WZQ is useful - might have a better build spec For one thing should change to use base alpine apk list libssl* # Search all libssh packages # Issue is the latest version of nginx doesn't work with this - using 1.21.0 instead Build works but image seems to not run properly docker run -it --rm nginxredis-nginx sh Had to change docker image to point to /nginx-1.21.0/objs/nginx - could clean this up better Compiled nginx points to /usr/local/nginx/html not in /usr/share - changed compose file But where is the config folder? Seems like I'm again pointing to the wrong place Just added to the build process to "include conf.d" - see if that works Had to move installs around - did dual image thing with separate build/run - had missing deps Wow that actually all worked. So I've got a 17MB image for running nginx with redis Might link it properly so it can run as you'd usually expect though (from /etc/nginx) Actually might make more sense to just configure it properly https://www.linuxcapable.com/how-to-build-nginx-from-source-on-ubuntu-linux/ Okay after a long amount of trial and error it works - had to configure build parameters This points it to a certain folder to run on - so had to make sure those folders existed Now trying to run it with fixed config but getting this issue: nginx: [crit] pread() "/etc/nginx/conf.d" failed (21: Is a directory) # Ohh I see I put "include /etc/nginx/conf.d" instead of "conf.d/*" Tried to enable the redis bit but seems to have an issue - might have to include the module RUN echo 'load_module modules/ngx_http_redis_module.so;' > /usr/local/nginx/conf/modules.conf After some some sed scripting to get things to load as I expect I might have got it to work Well it's accepting the config now (after ensuring folders exist) but not showing anything Guess I have to attach it to something to view - that's all I'm missing echo -n "Hello world" | redis-cli -h IP -x set /test.txt # Pipe data directly in to redis cat file.html | redis-cli -h 127.21.0.3 -x set /index.html Looks like redis isn't reading the config as I'd expect. I forgot to point to the config file Now seems to be running. I'll try cat the html file to that now - auth required... set env? # -a to pass raw password cat file.html | redis-cli -a thisismypassword -h 127.21.0.3 -x set /index.html redis-cli -a thisismypassword SCAN 0 # Get all data FLUSHDB # Delete everything Can't seem to get anything from nginx Might be worth installing redis-cli on nginx and seeing what data it gets "/var/www/html/html/index.html" is not found I changed the build to point to /var/www not /var/www/html and it's 403ing - oh double slash Remove trailing / from /var/www in Dockerfile and REBUILD Still 403 - but now not pointing to index.html - see if nginx can be persuaded Still nothing sent to redis - retiring for now. docker compose down # Shut down containers docker rmi -f $(docker images -q) # Deleting all images
2024-01-18 08:45 Going to try changing this to a redis nginx codebase I found yesterday - uses APIv2 https://github.com/openresty/redis2-nginx-module - pointing to this codebase instead now Weirdly that worked without issue Doesn't seem like it's reading in this config file as I want for nginx either Might just scrap the idea and run one of my own full nginx configs I think changing that /var/www/html might have confused things too - going with a clean option Also how can I get nginx logs outputted to docker logs? Suppose I could add a script to run nginx and then track the log file - not clean See now nginx is just exiting for no reason - add "stdin:open" and "tty:true" to compose.yaml docker compose run --rm nginx sh # Open a shell with docker compose # I put "redis" not "redis2" Okay getting better - now saying bad gateway Wow I just redirected it differently and now it's outputting a file with these contents: +OK $-1 Not ideal but something. Added index.html to redis. Let's see what that does +OK $-1 <html>testfile</html> # better default_type text/html # Add this to nginx config and it fixes most of it So the IP for redis keeps changing - need some fixed way of reading that Already using a network - but maybe not set correctly - use docker container name? Works fine now - but since I added back passwords that's saying noauth - got an idea redis2_query auth thisismypassword; Works perfectly now. How can I get rid of that "+OK +OK" Added "daemonize yes" and "save 30 1" to the config but seems to crash - daemonize crashing it Might be able to do this by using raw commands in nginx instead Okay I've moved a html file into redis/data - going to add a command to run to copy that Actually no I'm going to host that page with nginx and then try to save on redis end Could do this by running a script in nginx docker to keep updating redis like as before echo -n "Hello world" | redis-cli -h IP -x set /test.txt # Pipe directly, run this in a loop cat index.html | redis-cli -a thisismypassword \ -h nginxredis-redis-1 -x set /index.html Actually aside from the +OK this works perfectly Adding it to a loop now - need change image to also include redis-cli Seems to work fine. Let's see if it responds to changes Ohh so to get docker to run a command persistently don't use RUN use CMD or ENTRYPOINT CMD ["/bin/sh". "-c", "/usr/local/bin/redis-backup.sh; nginx -g daemon off"] # Could make this cleaner by combining into one command file - actually will do that COPY ./bin/nginx-redis /usr/local/bin/ RUN /usr/local/bin/nginx-redis # Later removable as it's run directly from CMD CMD ["/usr/local/bin/nginx-redis"] As a note if you're running nginx -g directly put "damon off;" in quotes nginx -g "daemon off;" CMD ["/usr/local/bin/nginx-redis"] Now I've got it on a 30 second cache of the website but it only serves one page - more? How could I do that? Could open it up to read from url - need something like nginx wildcards Apparently do this by defining a map - but this only works in http - need something better In the meantime nginx is taking a long time to restart each time - going to try fix that Going to separate out into redis-update container - that seems to work So turns out there's a grace period where it doesn't kill things - could lower that to 5 stop_grace_period: 5s # In compose.yaml (reduce destroy time) return 200 $html_file; # Use this to print out data from nginx directly # IN THE HTML section (capture FILENAME.html from /cache/FILENAME.html) map $uri $html_file { ~^/cache/(?P<file_name>.*\.html)$ $file_name; } # IN THE SERVER section location ~ ^/cache/[0-9a-z]+.html { redis2_raw_queries 2 "\r\nauth thisismypassword\r\nget /$html_file\r\n"; redis2_pass nginxredis-redis-1:6379; } That actually works perfectly. Not updating though - just need to update that script Oh right it's copying to "/cached/index.html" - just remove that and restart Now it works exactly as intended - got arbitrary files in / and arbitrary caching in /cache/ Now just going to increase that time to do caching and add a prompt to flushdb. Seems to randomly idle when I try to set variables Actually the issue is not ending raw queries in "\r\n" - no point using raw queries actually Store redis servers in an upstream list and you can use many at once # in HTML upstream redis_servers { server nginxredis-redis-1:6379; } # in LOCATION redis2_pass redis_servers; # e.g. location ~ ^/cache/[0-9a-z]+.html { redis2_query auth $redis_passwd; redis2_query GET /$html_file; redis2_pass redis_servers; } Now I can just make arbitrary pages for different redis actions I think this project is done. Would be interesting to see if I can run commands on any page... Or could use nginx subfilters to change contents of a page One last thing I'm going to do is see what latest version I can run nginx as to build this Cause currently it's using 1.21, which is a bit out of date - oh 1.25.3 works perfectly