Deleting unused kdb+ analytic files in RedHat

Hello guys :slight_smile:
I am new to using Unix and was hoping somebody could help me. Essentially, I am trying to clean out my database. For example, I have a directory filled with 100s of analytics and I want to know if they are used by any other analytics or if they are used by the front end user interface platforms. When I run

grep -rl (analytic_name)

it returns other analytics where this analytic is used. But i would like to run something that means i don't have to go through them one by one.
Thank you very much. Any help is appreciated

Linux 3.10.0-1062.1.2.el7.x86_64 #1 SMP Mon Sep 16 14:19:51 EDT 2019 x86_64 x86_64 x86_64 GNU/Linux

To ask these types of questions, you must provide your full system details, version numbers and application names and version number.

Is this what is needed?

Linux 3.10.0-1062.1.2.el7.x86_64 #1 SMP Mon Sep 16 14:19:51 EDT 2019 x86_64 x86_64 x86_64 GNU/Linux

You database application is?

Your exact Linux distribution ?

Where do I find this information? (I'm very new)

Try one of these methods:

  • a] /etc/*-release file.
  • b] lsb_release command
  • c] /proc/version file.
  • d] hostnamectl command

Normally, lsb_release is what I use.

Regarding deleting "database files" .... how can you consider deleting "database" files when you do not know what database application you are running?

Are you sure you mean "database" files? What kind of database, exactly?

1 Like

Regarding the version: lsb_release -a
The given kernel version only match for RHEL 7.7 or CentOS 7.7. Where the technical differences are neglectable.

The given grep command is one method to find out where references exist.

A method to check when the file was indeed last opened:
ls -lu filename

1 Like

When I run [lsb_release -a] I get:

[LSB Version:    :core-4.1-amd64:core-4.1-noarch]
[Distributor ID: RedHatEnterpriseServer]
[Description:    Red Hat Enterprise Linux Server release 7.7 (Maipo)]
[Release:        7.7]
[Codename:       Maipo]

I am using kdb+/q technology for the databases/UI's etc. Is this is what you are referring to? Thank you

OK. Good.

Before you can start deleting your DB files, you need to determine the recommended way kdb+/q recommends to truncate / trim the DB.

Do not do this by just removing files (yet); as this may not the proper way to truncate the DB tables (assume kdb+ has tables).

Have you looked into the docs on how to truncate tables (still assuming kdb+ has tables) in the DB?

Kdb+ for Developers - Limitless Big Data Applications | Kx

Please note, here is one of the pages of docs for this DB:

13. Commands and System Variables - Q for Mortals

You can use either q) or (if installed) a web based interface to manage kdb+

Have you ever done any DB admin tasks on this puppy?

Also, I assume you are familar with various IDEs to manage kdb+?

For example:

qStudio - A GUI for Kdb » qStudio Kdb+ IDE

Note, the docs state clearly:

How to you plan to manage this DB?

Have you reviewed the kdb+ FAQ?

Frequently-asked questions about kdb+ � Knowledge Base � kdb+ and q documentation

I wasn't planning on editing any of the tables, it was more deleting the analytics/functions that are not used to create these tables anymore.

eg) Analytic_X creates table A.

Then we add some additional logic to Analytic_X and create Analytic_Y.

Analytic_Y now creates Table A.

So I was hoping there was some command that retrieves the likes of Analytic_X that are not connected with anything so they can be removed? Hope that makes sense.

Yes, I understand.

In this FAQ, they discuss deleting rows in kdb+ tables:

Frequently-asked questions about kdb+ - Knowledge Base - kdb+ and q documentation

Do you know which tables you want to truncate / reduce in size?

See also:

System commands � Basics � kdb+ and q documentation

I don't want to truncate or reduce any tables. I want to delete analytic files that do not create any tables.

So from the backend I can go into the analytic directory where there are 100s of files (some create tables, some are old and need deleting). i don't want to touch the ones that create tables but delete the ones not connected to anything. I thought there might be a command to search for files that are used or not.

Yes, I think (did not check)... did MadeInGermany give you a command to check the last access times of files in his reply?

Or are you looking for something different or in addition?

I could be wrong, but I don't think "analytic files" (your words) "create tables" (your words).

Normally tables are created when the DB is created or modified by the admin, and "analytics files" add rows to those tables.

Is this not how your DB works?

Sorry, I don't have hands on experience with kdb+ but it looks interesting :slight_smile:

Also, I noticed (my guess) is that your application is related to predictive analytics and perhaps algorithmic trading.

In that case, these algorithms rely heavily on historical data and (often expensive) analytics.

I recommend you do not delete any of these "analytic files" unless you are sure you have backup copies.

May I ask why you are doing this (deleting analytical files which you believe to be unused)?

Are you having a disk space issue?